[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260809 07:31:25.526440 29021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.28.87.126:36317
I20260809 07:31:25.527837 29021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:31:25.528896 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:25.550040 29037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:25.550299 29030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:25.551823 29021 server_base.cc:1034] running on GCE node
W20260809 07:31:25.550477 29029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:26.678920 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:26.679152 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:26.679299 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260686679284 us; error 0 us; skew 500 ppm
I20260809 07:31:26.680084 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:26.689565 29021 webserver.cc:460] Webserver started at http://127.28.87.126:42587/ using document root <none> and password file <none>
I20260809 07:31:26.690454 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:26.690645 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:26.691025 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:26.697269 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/master-0-root/instance:
uuid: "f0b527a1ae9446ba9d3df3edfbb772bc"
format_stamp: "Formatted at 2026-08-09 07:31:26 on dist-test-slave-fgb3"
I20260809 07:31:26.705027 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.001s
I20260809 07:31:26.710423 29044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:26.711508 29021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:31:26.711833 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/master-0-root
uuid: "f0b527a1ae9446ba9d3df3edfbb772bc"
format_stamp: "Formatted at 2026-08-09 07:31:26 on dist-test-slave-fgb3"
I20260809 07:31:26.712162 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:26.756632 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:26.758177 29021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:31:26.758666 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:26.847456 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.126:36317
I20260809 07:31:26.847591 29115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.126:36317 every 8 connection(s)
I20260809 07:31:26.855113 29116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:26.882066 29116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc: Bootstrap starting.
I20260809 07:31:26.888521 29116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:26.890637 29116 log.cc:824] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc: Log is configured to *not* fsync() on all Append() calls
I20260809 07:31:26.895569 29116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc: No bootstrap required, opened a new log
I20260809 07:31:26.921260 29116 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" member_type: VOTER }
I20260809 07:31:26.921916 29116 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:26.922163 29116 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b527a1ae9446ba9d3df3edfbb772bc, State: Initialized, Role: FOLLOWER
I20260809 07:31:26.923069 29116 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" member_type: VOTER }
I20260809 07:31:26.923581 29116 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:31:26.923985 29116 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:31:26.924306 29116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:26.928802 29116 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" member_type: VOTER }
I20260809 07:31:26.929533 29116 leader_election.cc:302] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0b527a1ae9446ba9d3df3edfbb772bc; no voters:
I20260809 07:31:26.931914 29116 leader_election.cc:288] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:31:26.932188 29123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:26.935742 29123 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 1 LEADER]: Becoming Leader. State: Replica: f0b527a1ae9446ba9d3df3edfbb772bc, State: Running, Role: LEADER
I20260809 07:31:26.936903 29123 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" member_type: VOTER }
I20260809 07:31:26.938498 29116 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:31:26.955439 29124 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0b527a1ae9446ba9d3df3edfbb772bc. Latest consensus state: current_term: 1 leader_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" member_type: VOTER } }
I20260809 07:31:26.956197 29124 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [sys.catalog]: This master's current role is: LEADER
I20260809 07:31:26.958223 29125 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b527a1ae9446ba9d3df3edfbb772bc" member_type: VOTER } }
I20260809 07:31:26.958885 29125 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [sys.catalog]: This master's current role is: LEADER
I20260809 07:31:26.959396 29136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:31:26.970728 29136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:31:26.974412 29021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260809 07:31:26.986999 29136 catalog_manager.cc:1347] Generated new cluster ID: 45f33cc61cef45a8b96a67d621b1991d
I20260809 07:31:26.987298 29136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:31:27.017138 29136 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:31:27.019083 29136 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:31:27.033452 29136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc: Generated new TSK 0
I20260809 07:31:27.034420 29136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:31:27.045284 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:27.056783 29152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:27.060657 29157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:27.061369 29021 server_base.cc:1034] running on GCE node
W20260809 07:31:27.062844 29154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:27.063777 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:27.063983 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:27.064146 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260687064130 us; error 0 us; skew 500 ppm
I20260809 07:31:27.064719 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:27.068981 29021 webserver.cc:460] Webserver started at http://127.28.87.65:39627/ using document root <none> and password file <none>
I20260809 07:31:27.069547 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:27.069734 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:27.069993 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:27.071282 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-0-root/instance:
uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
format_stamp: "Formatted at 2026-08-09 07:31:27 on dist-test-slave-fgb3"
I20260809 07:31:27.076843 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 07:31:27.081017 29164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.081838 29021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:31:27.082178 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-0-root
uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
format_stamp: "Formatted at 2026-08-09 07:31:27 on dist-test-slave-fgb3"
I20260809 07:31:27.082599 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:27.103031 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:27.104727 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:27.107828 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:31:27.112356 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:31:27.112593 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.112861 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:31:27.113013 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.216195 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.65:41199
I20260809 07:31:27.216284 29261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.65:41199 every 8 connection(s)
I20260809 07:31:27.223531 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:27.232820 29269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:27.237076 29270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:27.248286 29021 server_base.cc:1034] running on GCE node
W20260809 07:31:27.248168 29273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:27.249733 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:27.249930 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:27.250110 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260687250096 us; error 0 us; skew 500 ppm
I20260809 07:31:27.250715 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:27.252641 29262 heartbeater.cc:347] Connected to a master server at 127.28.87.126:36317
I20260809 07:31:27.253121 29262 heartbeater.cc:464] Registering TS with master...
I20260809 07:31:27.253640 29021 webserver.cc:460] Webserver started at http://127.28.87.66:33079/ using document root <none> and password file <none>
I20260809 07:31:27.254287 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:27.254246 29262 heartbeater.cc:511] Master 127.28.87.126:36317 requested a full tablet report, sending...
I20260809 07:31:27.254609 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:27.254943 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:27.256253 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-1-root/instance:
uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
format_stamp: "Formatted at 2026-08-09 07:31:27 on dist-test-slave-fgb3"
I20260809 07:31:27.257766 29070 ts_manager.cc:194] Registered new tserver with Master: 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199)
I20260809 07:31:27.261147 29070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43456
I20260809 07:31:27.261685 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 07:31:27.266350 29279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.267171 29021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:31:27.267424 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-1-root
uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
format_stamp: "Formatted at 2026-08-09 07:31:27 on dist-test-slave-fgb3"
I20260809 07:31:27.267683 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:27.289280 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:27.290618 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:27.293102 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:31:27.296072 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:31:27.296267 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.296504 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:31:27.296650 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.395046 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.66:37527
I20260809 07:31:27.395248 29375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.66:37527 every 8 connection(s)
I20260809 07:31:27.405449 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:27.412943 29381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:27.416846 29383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:27.420830 29376 heartbeater.cc:347] Connected to a master server at 127.28.87.126:36317
I20260809 07:31:27.421260 29376 heartbeater.cc:464] Registering TS with master...
I20260809 07:31:27.422154 29376 heartbeater.cc:511] Master 127.28.87.126:36317 requested a full tablet report, sending...
I20260809 07:31:27.424805 29070 ts_manager.cc:194] Registered new tserver with Master: 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
W20260809 07:31:27.424875 29386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:27.425295 29021 server_base.cc:1034] running on GCE node
I20260809 07:31:27.426584 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:27.426856 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:27.426857 29070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43458
I20260809 07:31:27.427227 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260687427209 us; error 0 us; skew 500 ppm
I20260809 07:31:27.427987 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:27.431023 29021 webserver.cc:460] Webserver started at http://127.28.87.67:43815/ using document root <none> and password file <none>
I20260809 07:31:27.431674 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:27.431944 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:27.432250 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:27.433811 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-2-root/instance:
uuid: "0b56b8ed78214416a190e31e38e793ae"
format_stamp: "Formatted at 2026-08-09 07:31:27 on dist-test-slave-fgb3"
I20260809 07:31:27.439930 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:31:27.444087 29396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.444855 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:31:27.445192 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-2-root
uuid: "0b56b8ed78214416a190e31e38e793ae"
format_stamp: "Formatted at 2026-08-09 07:31:27 on dist-test-slave-fgb3"
I20260809 07:31:27.445550 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786260685288232-29021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:27.473799 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:27.475819 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:27.477663 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:31:27.480679 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:31:27.480949 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.481230 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:31:27.481448 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:27.576542 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.67:46033
I20260809 07:31:27.576591 29491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.67:46033 every 8 connection(s)
I20260809 07:31:27.593355 29493 heartbeater.cc:347] Connected to a master server at 127.28.87.126:36317
I20260809 07:31:27.593827 29493 heartbeater.cc:464] Registering TS with master...
I20260809 07:31:27.594713 29493 heartbeater.cc:511] Master 127.28.87.126:36317 requested a full tablet report, sending...
I20260809 07:31:27.596668 29070 ts_manager.cc:194] Registered new tserver with Master: 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:27.597044 29021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015856354s
I20260809 07:31:27.598632 29070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43460
I20260809 07:31:27.600776 29021 test_util.cc:268] Using random seed: 969063531
I20260809 07:31:27.655064 29070 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43464:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:31:27.658736 29070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:31:27.775065 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 93670b2e26c7405ca3bc86729c953acc (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260809 07:31:27.783147 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93670b2e26c7405ca3bc86729c953acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.792760 29199 tablet_service.cc:1467] Processing CreateTablet for tablet ff6bcc74c7c04e57bcfe6a4f5e01c305 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260809 07:31:27.794214 29206 tablet_service.cc:1467] Processing CreateTablet for tablet 70db551eedde440193453f228c8b3d27 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260809 07:31:27.791188 29439 tablet_service.cc:1467] Processing CreateTablet for tablet 7dda79df745442e7a5c99fd7d7bb00a2 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260809 07:31:27.790678 29434 tablet_service.cc:1467] Processing CreateTablet for tablet 21bbdfafc1344a22a8e1acf1da4b2846 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260809 07:31:27.798061 29199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bcc74c7c04e57bcfe6a4f5e01c305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.792773 29438 tablet_service.cc:1467] Processing CreateTablet for tablet 8feaae344637446fbbd0330ecb39054f (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260809 07:31:27.791872 29193 tablet_service.cc:1467] Processing CreateTablet for tablet 4c336377b4204dca9c20db902c911a95 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260809 07:31:27.802492 29202 tablet_service.cc:1467] Processing CreateTablet for tablet 8feaae344637446fbbd0330ecb39054f (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260809 07:31:27.802520 29197 tablet_service.cc:1467] Processing CreateTablet for tablet 21bbdfafc1344a22a8e1acf1da4b2846 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260809 07:31:27.802836 29439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dda79df745442e7a5c99fd7d7bb00a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.792996 29435 tablet_service.cc:1467] Processing CreateTablet for tablet 46df14febbe54742affebecfe152b06a (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260809 07:31:27.798986 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 93670b2e26c7405ca3bc86729c953acc (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260809 07:31:27.800711 29325 tablet_service.cc:1467] Processing CreateTablet for tablet 93670b2e26c7405ca3bc86729c953acc (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260809 07:31:27.809698 29438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feaae344637446fbbd0330ecb39054f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.796996 29436 tablet_service.cc:1467] Processing CreateTablet for tablet ff6bcc74c7c04e57bcfe6a4f5e01c305 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260809 07:31:27.799356 29440 tablet_service.cc:1467] Processing CreateTablet for tablet 70db551eedde440193453f228c8b3d27 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260809 07:31:27.799080 29198 tablet_service.cc:1467] Processing CreateTablet for tablet 46df14febbe54742affebecfe152b06a (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260809 07:31:27.813189 29319 tablet_service.cc:1467] Processing CreateTablet for tablet 7dda79df745442e7a5c99fd7d7bb00a2 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260809 07:31:27.799032 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 5a01215676044eb9b3d46bd768aa4ebb (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260809 07:31:27.814292 29206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70db551eedde440193453f228c8b3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.804893 29205 tablet_service.cc:1467] Processing CreateTablet for tablet 7dda79df745442e7a5c99fd7d7bb00a2 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260809 07:31:27.819862 29315 tablet_service.cc:1467] Processing CreateTablet for tablet 8feaae344637446fbbd0330ecb39054f (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260809 07:31:27.817629 29434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21bbdfafc1344a22a8e1acf1da4b2846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.823715 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 21bbdfafc1344a22a8e1acf1da4b2846 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260809 07:31:27.826668 29306 tablet_service.cc:1467] Processing CreateTablet for tablet 5a01215676044eb9b3d46bd768aa4ebb (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260809 07:31:27.828488 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46df14febbe54742affebecfe152b06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.830020 29325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93670b2e26c7405ca3bc86729c953acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.818281 29433 tablet_service.cc:1467] Processing CreateTablet for tablet 5a01215676044eb9b3d46bd768aa4ebb (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260809 07:31:27.834151 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93670b2e26c7405ca3bc86729c953acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.832799 29202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feaae344637446fbbd0330ecb39054f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.839504 29309 tablet_service.cc:1467] Processing CreateTablet for tablet 46df14febbe54742affebecfe152b06a (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260809 07:31:27.841543 29205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dda79df745442e7a5c99fd7d7bb00a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.841470 29305 tablet_service.cc:1467] Processing CreateTablet for tablet 4c336377b4204dca9c20db902c911a95 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260809 07:31:27.834877 29429 tablet_service.cc:1467] Processing CreateTablet for tablet 4c336377b4204dca9c20db902c911a95 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260809 07:31:27.830515 29324 tablet_service.cc:1467] Processing CreateTablet for tablet 70db551eedde440193453f228c8b3d27 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260809 07:31:27.845741 29440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70db551eedde440193453f228c8b3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.838228 29313 tablet_service.cc:1467] Processing CreateTablet for tablet ff6bcc74c7c04e57bcfe6a4f5e01c305 (DEFAULT_TABLE table=test-workload [id=f7122d961ca3481ca357306578c03163]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260809 07:31:27.847133 29193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c336377b4204dca9c20db902c911a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.846985 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dda79df745442e7a5c99fd7d7bb00a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.854703 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bcc74c7c04e57bcfe6a4f5e01c305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.855355 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feaae344637446fbbd0330ecb39054f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.849555 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a01215676044eb9b3d46bd768aa4ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.859103 29309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46df14febbe54742affebecfe152b06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.860952 29198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46df14febbe54742affebecfe152b06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.865633 29305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c336377b4204dca9c20db902c911a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.867895 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bcc74c7c04e57bcfe6a4f5e01c305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.871675 29197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21bbdfafc1344a22a8e1acf1da4b2846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.877786 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21bbdfafc1344a22a8e1acf1da4b2846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.879189 29429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c336377b4204dca9c20db902c911a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.889467 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a01215676044eb9b3d46bd768aa4ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.895117 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70db551eedde440193453f228c8b3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:27.899215 29306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a01215676044eb9b3d46bd768aa4ebb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 07:31:27.997910 29138 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:31:28.062853 29521 tablet_bootstrap.cc:492] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.135141 29520 tablet_bootstrap.cc:492] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:28.176047 29521 tablet_bootstrap.cc:654] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.234228 29520 tablet_bootstrap.cc:654] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.239509 29523 tablet_bootstrap.cc:492] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:28.243899 29520 tablet_bootstrap.cc:492] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:28.244280 29520 ts_tablet_manager.cc:1397] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.155s user 0.006s sys 0.007s
I20260809 07:31:28.244570 29521 tablet_bootstrap.cc:492] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.245150 29521 ts_tablet_manager.cc:1397] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.184s user 0.008s sys 0.020s
I20260809 07:31:28.246572 29520 raft_consensus.cc:348] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.247366 29520 raft_consensus.cc:374] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.247653 29520 raft_consensus.cc:729] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.248072 29523 tablet_bootstrap.cc:654] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.248441 29521 raft_consensus.cc:348] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.248337 29520 consensus_queue.cc:260] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.249155 29521 raft_consensus.cc:374] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.249456 29521 raft_consensus.cc:729] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.250370 29521 consensus_queue.cc:260] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.257463 29262 heartbeater.cc:503] Master 127.28.87.126:36317 was elected leader, sending a full tablet report...
I20260809 07:31:28.257511 29520 ts_tablet_manager.cc:1428] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260809 07:31:28.259456 29523 tablet_bootstrap.cc:492] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:28.259693 29521 ts_tablet_manager.cc:1428] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 07:31:28.259889 29523 ts_tablet_manager.cc:1397] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.004s
I20260809 07:31:28.260762 29493 heartbeater.cc:503] Master 127.28.87.126:36317 was elected leader, sending a full tablet report...
I20260809 07:31:28.261164 29521 tablet_bootstrap.cc:492] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.262688 29520 tablet_bootstrap.cc:492] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:28.269196 29523 raft_consensus.cc:348] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.269977 29523 raft_consensus.cc:374] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.270293 29523 raft_consensus.cc:729] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.270980 29523 consensus_queue.cc:260] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.271968 29521 tablet_bootstrap.cc:654] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.279695 29376 heartbeater.cc:503] Master 127.28.87.126:36317 was elected leader, sending a full tablet report...
I20260809 07:31:28.280588 29523 ts_tablet_manager.cc:1428] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.020s user 0.005s sys 0.009s
I20260809 07:31:28.281598 29523 tablet_bootstrap.cc:492] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:28.288300 29523 tablet_bootstrap.cc:654] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.297580 29520 tablet_bootstrap.cc:654] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.300544 29523 tablet_bootstrap.cc:492] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:28.300984 29523 ts_tablet_manager.cc:1397] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.008s
I20260809 07:31:28.303398 29521 tablet_bootstrap.cc:492] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.303848 29521 ts_tablet_manager.cc:1397] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.043s user 0.023s sys 0.014s
I20260809 07:31:28.303683 29523 raft_consensus.cc:348] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.304432 29523 raft_consensus.cc:374] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.304742 29523 raft_consensus.cc:729] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.305416 29523 consensus_queue.cc:260] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.306553 29521 raft_consensus.cc:348] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.307258 29521 raft_consensus.cc:374] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.307524 29521 raft_consensus.cc:729] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.307788 29523 ts_tablet_manager.cc:1428] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260809 07:31:28.308178 29521 consensus_queue.cc:260] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.308692 29523 tablet_bootstrap.cc:492] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:28.310475 29521 ts_tablet_manager.cc:1428] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:31:28.311452 29521 tablet_bootstrap.cc:492] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.317663 29521 tablet_bootstrap.cc:654] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.319082 29523 tablet_bootstrap.cc:654] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.327050 29520 tablet_bootstrap.cc:492] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:28.329245 29520 ts_tablet_manager.cc:1397] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.067s user 0.015s sys 0.000s
I20260809 07:31:28.329581 29521 tablet_bootstrap.cc:492] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.330132 29523 tablet_bootstrap.cc:492] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:28.330284 29521 ts_tablet_manager.cc:1397] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.003s
I20260809 07:31:28.330577 29523 ts_tablet_manager.cc:1397] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.013s
I20260809 07:31:28.333067 29523 raft_consensus.cc:348] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.333014 29521 raft_consensus.cc:348] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
W20260809 07:31:28.333738 29494 tablet.cc:2369] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:31:28.333839 29523 raft_consensus.cc:374] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.333906 29521 raft_consensus.cc:374] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.334275 29523 raft_consensus.cc:729] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.334342 29521 raft_consensus.cc:729] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.334764 29520 raft_consensus.cc:348] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.335060 29521 consensus_queue.cc:260] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.335050 29523 consensus_queue.cc:260] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.335686 29520 raft_consensus.cc:374] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.337530 29521 ts_tablet_manager.cc:1428] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 07:31:28.337530 29523 ts_tablet_manager.cc:1428] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260809 07:31:28.338586 29521 tablet_bootstrap.cc:492] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.338589 29523 tablet_bootstrap.cc:492] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:28.345394 29521 tablet_bootstrap.cc:654] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.345552 29523 tablet_bootstrap.cc:654] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.336141 29520 raft_consensus.cc:729] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.348239 29520 consensus_queue.cc:260] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.351528 29520 ts_tablet_manager.cc:1428] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260809 07:31:28.352417 29520 tablet_bootstrap.cc:492] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:28.357144 29521 tablet_bootstrap.cc:492] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.357882 29533 raft_consensus.cc:482] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.358144 29523 tablet_bootstrap.cc:492] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:28.358719 29523 ts_tablet_manager.cc:1397] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.011s
I20260809 07:31:28.358407 29533 raft_consensus.cc:504] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.361443 29533 leader_election.cc:288] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.357591 29521 ts_tablet_manager.cc:1397] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20260809 07:31:28.362221 29523 raft_consensus.cc:348] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.363091 29523 raft_consensus.cc:374] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.364768 29523 raft_consensus.cc:729] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.366746 29521 raft_consensus.cc:348] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.375646 29535 raft_consensus.cc:482] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.371826 29521 raft_consensus.cc:374] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.371145 29523 consensus_queue.cc:260] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.379580 29535 raft_consensus.cc:504] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.379721 29521 raft_consensus.cc:729] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.383364 29529 raft_consensus.cc:482] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.384860 29529 raft_consensus.cc:504] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.387789 29529 leader_election.cc:288] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.388432 29521 consensus_queue.cc:260] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.397279 29535 leader_election.cc:288] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.396759 29521 ts_tablet_manager.cc:1428] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260809 07:31:28.401899 29520 tablet_bootstrap.cc:654] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.402549 29529 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.406266 29521 tablet_bootstrap.cc:492] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.404446 29529 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.407797 29529 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.428493 29521 tablet_bootstrap.cc:654] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.452750 29523 ts_tablet_manager.cc:1428] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.094s user 0.019s sys 0.005s
I20260809 07:31:28.460975 29523 tablet_bootstrap.cc:492] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:28.473004 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c336377b4204dca9c20db902c911a95" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
W20260809 07:31:28.480132 29397 leader_election.cc:341] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.487109 29521 tablet_bootstrap.cc:492] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.490059 29520 tablet_bootstrap.cc:492] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:28.488943 29521 ts_tablet_manager.cc:1397] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.083s user 0.016s sys 0.009s
I20260809 07:31:28.488232 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c336377b4204dca9c20db902c911a95" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.492332 29340 raft_consensus.cc:2454] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:28.493326 29520 ts_tablet_manager.cc:1397] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.141s user 0.019s sys 0.017s
I20260809 07:31:28.498564 29554 raft_consensus.cc:482] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.499297 29554 raft_consensus.cc:504] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.510238 29401 leader_election.cc:302] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters: 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.511996 29533 raft_consensus.cc:2781] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:28.512573 29533 raft_consensus.cc:482] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:28.512890 29533 raft_consensus.cc:3037] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.512988 29554 leader_election.cc:288] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.515689 29521 raft_consensus.cc:348] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.516664 29521 raft_consensus.cc:374] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.517194 29521 raft_consensus.cc:729] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.519244 29521 consensus_queue.cc:260] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.523950 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.527263 29523 tablet_bootstrap.cc:654] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.527356 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.529699 29340 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 0.
W20260809 07:31:28.534219 29166 leader_election.cc:341] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.534868 29520 raft_consensus.cc:348] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.536453 29520 raft_consensus.cc:374] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.536664 29166 leader_election.cc:302] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:28.538057 29529 raft_consensus.cc:2781] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:28.536841 29520 raft_consensus.cc:729] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.538841 29529 raft_consensus.cc:482] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:28.539526 29529 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.529197 29533 raft_consensus.cc:504] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.546445 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:28.549611 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
W20260809 07:31:28.552773 29280 leader_election.cc:341] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.555164 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c336377b4204dca9c20db902c911a95" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
W20260809 07:31:28.557238 29397 leader_election.cc:341] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.561115 29533 leader_election.cc:288] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.542685 29520 consensus_queue.cc:260] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.559463 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:28.566663 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:28.568224 29458 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 0.
I20260809 07:31:28.572347 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c336377b4204dca9c20db902c911a95" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:28.574579 29340 raft_consensus.cc:3037] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:31:28.578516 29168 leader_election.cc:341] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.579276 29168 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89ed39c1f88b4fb18cb9b5712dcaa026; no voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:28.582538 29521 ts_tablet_manager.cc:1428] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.090s user 0.002s sys 0.015s
I20260809 07:31:28.584005 29521 tablet_bootstrap.cc:492] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.589293 29529 raft_consensus.cc:504] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.591912 29225 raft_consensus.cc:2379] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015d82d3a9824f689d3b0a54aa41e5cb in current term 1: Already voted for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in this term.
I20260809 07:31:28.596195 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:28.596963 29338 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.593592 29529 leader_election.cc:288] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.609411 29529 raft_consensus.cc:2726] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:28.615231 29570 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.615978 29570 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.623121 29570 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.624998 29520 ts_tablet_manager.cc:1428] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.128s user 0.013s sys 0.011s
W20260809 07:31:28.624755 29168 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:46033, user_credentials={real_user=slave}} blocked reactor thread for 52013.9us
I20260809 07:31:28.627132 29523 tablet_bootstrap.cc:492] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:28.629055 29554 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.628803 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.629621 29523 ts_tablet_manager.cc:1397] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.169s user 0.025s sys 0.017s
I20260809 07:31:28.629909 29554 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.631188 29340 raft_consensus.cc:2454] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:28.631531 29337 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:28.632885 29401 leader_election.cc:302] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters: 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.633880 29554 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.635732 29520 tablet_bootstrap.cc:492] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:28.634739 29401 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:28.635838 29570 raft_consensus.cc:2781] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:28.637075 29533 raft_consensus.cc:2781] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:28.693020 29521 tablet_bootstrap.cc:654] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.638101 29533 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:28.638461 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:28.647784 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:28.703231 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:28.703068 29523 raft_consensus.cc:348] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.712245 29533 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.712962 29575 raft_consensus.cc:482] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.712204 29570 raft_consensus.cc:686] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:28.712613 29338 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 1.
I20260809 07:31:28.714465 29523 raft_consensus.cc:374] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.716322 29572 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.714237 29458 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.716539 29574 raft_consensus.cc:482] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.717428 29523 raft_consensus.cc:729] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.716228 29575 raft_consensus.cc:504] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.721623 29166 leader_election.cc:302] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:28.719074 29572 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.727236 29573 raft_consensus.cc:2781] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:28.721057 29574 raft_consensus.cc:504] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.724726 29523 consensus_queue.cc:260] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.728725 29573 raft_consensus.cc:686] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:28.728132 29225 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:28.724591 29570 consensus_queue.cc:237] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.741111 29575 leader_election.cc:288] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.740209 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:28.744320 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.750721 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
W20260809 07:31:28.750792 29401 leader_election.cc:341] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.752538 29338 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 0.
I20260809 07:31:28.753831 29166 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:28.742192 29573 consensus_queue.cc:237] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.757416 29529 raft_consensus.cc:2781] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:28.757653 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:28.756049 29572 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.757997 29574 leader_election.cc:288] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.759212 29529 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:28.762089 29529 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.763885 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
W20260809 07:31:28.761166 29397 leader_election.cc:341] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.768692 29397 leader_election.cc:302] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae; no voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.760514 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.771829 29338 raft_consensus.cc:2454] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:28.777011 29523 ts_tablet_manager.cc:1428] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.146s user 0.010s sys 0.006s
I20260809 07:31:28.777293 29455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:28.776211 29533 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.778991 29574 raft_consensus.cc:2726] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:28.779054 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:28.785512 29523 tablet_bootstrap.cc:492] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
W20260809 07:31:28.783140 29397 leader_election.cc:341] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.786586 29457 raft_consensus.cc:2379] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in current term 1: Already voted for candidate 0b56b8ed78214416a190e31e38e793ae in this term.
I20260809 07:31:28.787405 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:28.790494 29284 leader_election.cc:302] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb; no voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.788707 29397 leader_election.cc:302] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters: 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.796737 29554 raft_consensus.cc:2726] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:31:28.798293 29284 leader_election.cc:341] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.799062 29284 leader_election.cc:302] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb; no voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.800144 29458 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 1.
I20260809 07:31:28.800200 29554 raft_consensus.cc:2726] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:28.800316 29533 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.801646 29284 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb; no voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.802641 29554 raft_consensus.cc:2726] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:28.806890 29520 tablet_bootstrap.cc:654] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.807508 29529 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.814805 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:28.816269 29338 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.816212 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:28.815374 29521 tablet_bootstrap.cc:492] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.814343 29529 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.817602 29521 ts_tablet_manager.cc:1397] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.234s user 0.016s sys 0.018s
I20260809 07:31:28.831491 29520 tablet_bootstrap.cc:492] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:28.832422 29520 ts_tablet_manager.cc:1397] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.198s user 0.014s sys 0.017s
I20260809 07:31:28.832489 29521 raft_consensus.cc:348] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.834184 29521 raft_consensus.cc:374] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.834786 29521 raft_consensus.cc:729] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.837118 29338 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 1.
I20260809 07:31:28.838459 29166 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters: 0b56b8ed78214416a190e31e38e793ae
W20260809 07:31:28.838830 29401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:37527, user_credentials={real_user=slave}} blocked reactor thread for 54266.3us
I20260809 07:31:28.840783 29523 tablet_bootstrap.cc:654] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.841780 29529 raft_consensus.cc:2781] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:28.842517 29529 raft_consensus.cc:686] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:28.836596 29521 consensus_queue.cc:260] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.843655 29529 consensus_queue.cc:237] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.845415 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:28.849221 29521 ts_tablet_manager.cc:1428] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.031s user 0.010s sys 0.004s
I20260809 07:31:28.846091 29570 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.849960 29570 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.850698 29521 tablet_bootstrap.cc:492] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:28.846057 29340 raft_consensus.cc:2379] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0b56b8ed78214416a190e31e38e793ae in current term 1: Already voted for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in this term.
W20260809 07:31:28.857760 29397 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:41199, user_credentials={real_user=slave}} blocked reactor thread for 76166.1us
I20260809 07:31:28.858940 29533 raft_consensus.cc:2781] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:28.859335 29533 raft_consensus.cc:482] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:28.859853 29533 raft_consensus.cc:3037] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.860270 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:28.862600 29397 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae; no voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.864554 29574 raft_consensus.cc:2726] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:31:28.862995 29070 catalog_manager.cc:5495] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 1 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:28.838990 29520 raft_consensus.cc:348] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.872042 29069 catalog_manager.cc:5495] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:28.878134 29520 raft_consensus.cc:374] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.878911 29533 raft_consensus.cc:504] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.883349 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:28.884188 29340 raft_consensus.cc:3037] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.884317 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:28.885040 29523 tablet_bootstrap.cc:492] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:28.885658 29523 ts_tablet_manager.cc:1397] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.102s user 0.017s sys 0.010s
W20260809 07:31:28.885700 29397 leader_election.cc:341] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.886380 29533 leader_election.cc:288] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.889482 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:28.890298 29224 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:28.891848 29397 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:28.893033 29533 raft_consensus.cc:2781] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:28.882074 29520 raft_consensus.cc:729] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.893534 29533 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:28.894738 29340 raft_consensus.cc:2454] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:28.895339 29533 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.895746 29570 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.897583 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:28.897321 29520 consensus_queue.cc:260] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.900271 29521 tablet_bootstrap.cc:654] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.900843 29401 leader_election.cc:302] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters: 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:28.901968 29520 ts_tablet_manager.cc:1428] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.069s user 0.005s sys 0.005s
W20260809 07:31:28.902801 29401 leader_election.cc:341] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.904160 29570 raft_consensus.cc:2781] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:28.904500 29523 raft_consensus.cc:348] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.905346 29570 raft_consensus.cc:686] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:28.905663 29523 raft_consensus.cc:374] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:28.906172 29523 raft_consensus.cc:729] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:28.906746 29570 consensus_queue.cc:237] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.907333 29523 consensus_queue.cc:260] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.903149 29520 tablet_bootstrap.cc:492] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:28.912611 29533 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.931597 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:28.932397 29224 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:28.932578 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
W20260809 07:31:28.934414 29401 leader_election.cc:341] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.943781 29224 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:28.945082 29397 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters: 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:28.945802 29520 tablet_bootstrap.cc:654] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.946470 29574 raft_consensus.cc:2781] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:28.922206 29533 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:28.953728 29529 raft_consensus.cc:482] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:28.954180 29529 raft_consensus.cc:504] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.955353 29523 ts_tablet_manager.cc:1428] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.069s user 0.002s sys 0.012s
I20260809 07:31:28.957854 29523 tablet_bootstrap.cc:492] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:28.952768 29574 raft_consensus.cc:686] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:28.960713 29574 consensus_queue.cc:237] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.964018 29529 leader_election.cc:288] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:28.971956 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
W20260809 07:31:28.973665 29166 leader_election.cc:341] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.975994 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
W20260809 07:31:28.983353 29168 leader_election.cc:341] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:28.983687 29523 tablet_bootstrap.cc:654] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:28.984035 29168 leader_election.cc:302] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89ed39c1f88b4fb18cb9b5712dcaa026; no voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:28.985361 29529 raft_consensus.cc:2726] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:28.986923 29521 tablet_bootstrap.cc:492] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:28.987505 29521 ts_tablet_manager.cc:1397] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.137s user 0.023s sys 0.016s
I20260809 07:31:28.994656 29521 raft_consensus.cc:348] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:28.995540 29521 raft_consensus.cc:374] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.000023 29520 tablet_bootstrap.cc:492] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:29.000883 29520 ts_tablet_manager.cc:1397] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.098s user 0.009s sys 0.036s
I20260809 07:31:28.996922 29521 raft_consensus.cc:729] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.008002 29520 raft_consensus.cc:348] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.009313 29520 raft_consensus.cc:374] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.009816 29520 raft_consensus.cc:729] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.010717 29521 consensus_queue.cc:260] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.010743 29520 consensus_queue.cc:260] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
W20260809 07:31:29.016813 29138 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:31:29.021904 29520 ts_tablet_manager.cc:1428] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.021s user 0.003s sys 0.006s
I20260809 07:31:29.022110 29523 tablet_bootstrap.cc:492] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:29.022933 29523 ts_tablet_manager.cc:1397] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.014s
I20260809 07:31:29.025506 29070 catalog_manager.cc:5495] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:29.027084 29069 catalog_manager.cc:5495] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 1 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:29.030637 29520 tablet_bootstrap.cc:492] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:29.029912 29069 catalog_manager.cc:5495] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 1 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:29.040829 29521 ts_tablet_manager.cc:1428] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.053s user 0.007s sys 0.008s
I20260809 07:31:29.043202 29521 tablet_bootstrap.cc:492] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:29.052407 29520 tablet_bootstrap.cc:654] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.064069 29523 raft_consensus.cc:348] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.065047 29523 raft_consensus.cc:374] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.065407 29523 raft_consensus.cc:729] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.066149 29523 consensus_queue.cc:260] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.071761 29521 tablet_bootstrap.cc:654] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.072858 29523 ts_tablet_manager.cc:1428] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.049s user 0.004s sys 0.007s
I20260809 07:31:29.074259 29523 tablet_bootstrap.cc:492] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:29.077036 29520 tablet_bootstrap.cc:492] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:29.077723 29520 ts_tablet_manager.cc:1397] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.021s
I20260809 07:31:29.083735 29520 raft_consensus.cc:348] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.087842 29523 tablet_bootstrap.cc:654] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.085582 29520 raft_consensus.cc:374] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.090515 29520 raft_consensus.cc:729] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.091252 29520 consensus_queue.cc:260] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.099625 29520 ts_tablet_manager.cc:1428] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.021s user 0.007s sys 0.007s
I20260809 07:31:29.100175 29523 tablet_bootstrap.cc:492] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:29.100870 29523 ts_tablet_manager.cc:1397] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.011s
I20260809 07:31:29.101056 29520 tablet_bootstrap.cc:492] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:29.107539 29523 raft_consensus.cc:348] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.108331 29523 raft_consensus.cc:374] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.108596 29523 raft_consensus.cc:729] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.109304 29523 consensus_queue.cc:260] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.111936 29523 ts_tablet_manager.cc:1428] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.011s user 0.000s sys 0.008s
I20260809 07:31:29.112906 29523 tablet_bootstrap.cc:492] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: Bootstrap starting.
I20260809 07:31:29.115757 29520 tablet_bootstrap.cc:654] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.120826 29521 tablet_bootstrap.cc:492] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:29.121439 29521 ts_tablet_manager.cc:1397] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.079s user 0.007s sys 0.019s
I20260809 07:31:29.124744 29523 tablet_bootstrap.cc:654] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.125249 29521 raft_consensus.cc:348] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.125949 29521 raft_consensus.cc:374] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.126226 29521 raft_consensus.cc:729] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.127018 29521 consensus_queue.cc:260] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.129395 29521 ts_tablet_manager.cc:1428] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.008s user 0.006s sys 0.003s
I20260809 07:31:29.131151 29521 tablet_bootstrap.cc:492] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: Bootstrap starting.
I20260809 07:31:29.141134 29520 tablet_bootstrap.cc:492] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:29.141842 29520 ts_tablet_manager.cc:1397] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.014s
I20260809 07:31:29.143731 29521 tablet_bootstrap.cc:654] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.147061 29574 raft_consensus.cc:482] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:29.147748 29574 raft_consensus.cc:504] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.151886 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:29.152815 29224 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:29.154362 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:29.156313 29397 leader_election.cc:302] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:29.156960 29523 tablet_bootstrap.cc:492] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: No bootstrap required, opened a new log
I20260809 07:31:29.157382 29523 ts_tablet_manager.cc:1397] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.015s
I20260809 07:31:29.155112 29338 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:29.157465 29570 raft_consensus.cc:2781] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:29.157924 29574 leader_election.cc:288] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:29.158243 29570 raft_consensus.cc:482] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:29.158655 29570 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.160274 29523 raft_consensus.cc:348] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.160949 29523 raft_consensus.cc:374] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.161269 29523 raft_consensus.cc:729] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.161965 29523 consensus_queue.cc:260] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.164366 29523 ts_tablet_manager.cc:1428] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260809 07:31:29.166239 29570 raft_consensus.cc:504] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.167716 29570 leader_election.cc:288] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:29.157472 29520 raft_consensus.cc:348] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.170593 29520 raft_consensus.cc:374] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.170910 29520 raft_consensus.cc:729] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.171988 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:29.171613 29520 consensus_queue.cc:260] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.172570 29338 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.174618 29520 ts_tablet_manager.cc:1428] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.032s user 0.007s sys 0.005s
I20260809 07:31:29.178944 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:29.179510 29225 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:31:29.187670 29397 consensus_peers.cc:487] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae -> Peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Couldn't send request to peer 89ed39c1f88b4fb18cb9b5712dcaa026. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:31:29.189090 29225 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:29.190145 29520 tablet_bootstrap.cc:492] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:29.191390 29338 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:29.192565 29401 leader_election.cc:302] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:29.197609 29520 tablet_bootstrap.cc:654] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.198735 29570 raft_consensus.cc:2781] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:29.199958 29570 raft_consensus.cc:686] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:29.200785 29570 consensus_queue.cc:237] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.205138 29521 tablet_bootstrap.cc:492] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: No bootstrap required, opened a new log
I20260809 07:31:29.205602 29521 ts_tablet_manager.cc:1397] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: Time spent bootstrapping tablet: real 0.075s user 0.026s sys 0.028s
I20260809 07:31:29.208273 29521 raft_consensus.cc:348] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.209004 29521 raft_consensus.cc:374] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.209260 29521 raft_consensus.cc:729] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.209539 29529 consensus_queue.cc:1035] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.209887 29521 consensus_queue.cc:260] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.213052 29521 ts_tablet_manager.cc:1428] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260809 07:31:29.215112 29520 tablet_bootstrap.cc:492] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:29.215547 29520 ts_tablet_manager.cc:1397] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.006s
I20260809 07:31:29.217602 29520 raft_consensus.cc:348] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.218241 29520 raft_consensus.cc:374] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.218552 29520 raft_consensus.cc:729] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.218369 29065 catalog_manager.cc:5495] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 1 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:29.219178 29520 consensus_queue.cc:260] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.222960 29570 consensus_queue.cc:1035] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.223790 29520 ts_tablet_manager.cc:1428] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20260809 07:31:29.224629 29520 tablet_bootstrap.cc:492] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026: Bootstrap starting.
I20260809 07:31:29.238176 29520 tablet_bootstrap.cc:654] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:29.260530 29572 raft_consensus.cc:482] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:29.261062 29572 raft_consensus.cc:504] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.261855 29573 consensus_queue.cc:1035] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:29.263631 29572 leader_election.cc:288] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:29.265829 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:29.266651 29338 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 0.
I20260809 07:31:29.268218 29166 leader_election.cc:302] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:29.270342 29572 raft_consensus.cc:2781] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:29.272189 29572 raft_consensus.cc:482] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:29.272891 29572 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.274889 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:29.275853 29457 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 0.
I20260809 07:31:29.278517 29520 tablet_bootstrap.cc:492] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026: No bootstrap required, opened a new log
I20260809 07:31:29.279021 29520 ts_tablet_manager.cc:1397] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent bootstrapping tablet: real 0.055s user 0.022s sys 0.000s
I20260809 07:31:29.285435 29529 consensus_queue.cc:1035] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:29.288518 29520 raft_consensus.cc:348] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.289248 29520 raft_consensus.cc:374] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:29.289500 29520 raft_consensus.cc:729] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Initialized, Role: FOLLOWER
I20260809 07:31:29.290064 29520 consensus_queue.cc:260] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.321939 29574 raft_consensus.cc:482] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:29.322954 29574 raft_consensus.cc:504] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.327307 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:29.328073 29574 leader_election.cc:288] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:29.328236 29225 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:29.329746 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:29.333699 29397 leader_election.cc:302] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:29.335088 29337 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:29.338646 29574 raft_consensus.cc:2781] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:29.339197 29574 raft_consensus.cc:482] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:29.339618 29574 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.351123 29573 consensus_queue.cc:1035] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:29.370437 29572 raft_consensus.cc:504] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.378057 29573 raft_consensus.cc:482] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:29.378399 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:29.379346 29340 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.379053 29573 raft_consensus.cc:504] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.391351 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c336377b4204dca9c20db902c911a95" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:29.393639 29520 ts_tablet_manager.cc:1428] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026: Time spent starting tablet: real 0.114s user 0.011s sys 0.033s
I20260809 07:31:29.393551 29570 raft_consensus.cc:482] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:29.394723 29572 leader_election.cc:288] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:29.373708 29574 raft_consensus.cc:504] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.394279 29570 raft_consensus.cc:504] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.399065 29570 leader_election.cc:288] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:29.401134 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" is_pre_election: true
I20260809 07:31:29.401587 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:29.401724 29455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:29.402091 29337 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:29.402540 29455 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.401134 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c336377b4204dca9c20db902c911a95" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:29.409700 29401 leader_election.cc:302] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:29.411180 29168 leader_election.cc:302] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89ed39c1f88b4fb18cb9b5712dcaa026; no voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:29.402550 29225 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 0.
I20260809 07:31:29.393639 29573 leader_election.cc:288] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:29.416105 29570 raft_consensus.cc:2781] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:29.420310 29570 raft_consensus.cc:482] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:29.420428 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:29.420883 29570 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.421272 29225 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.421136 29340 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 1.
I20260809 07:31:29.418641 29572 raft_consensus.cc:2726] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:29.424108 29166 leader_election.cc:302] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:29.426676 29340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:29.427536 29340 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.431763 29574 leader_election.cc:288] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:29.433668 29570 raft_consensus.cc:504] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.435936 29629 raft_consensus.cc:2781] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:29.436380 29629 raft_consensus.cc:686] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:29.437204 29629 consensus_queue.cc:237] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.438342 29225 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:29.439756 29397 leader_election.cc:302] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:29.440819 29574 raft_consensus.cc:2781] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:29.441452 29574 raft_consensus.cc:686] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:29.442214 29574 consensus_queue.cc:237] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.446585 29554 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:29.447062 29554 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.451808 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" is_pre_election: true
I20260809 07:31:29.452589 29224 raft_consensus.cc:2379] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015d82d3a9824f689d3b0a54aa41e5cb in current term 1: Already voted for candidate 0b56b8ed78214416a190e31e38e793ae in this term.
I20260809 07:31:29.454483 29554 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:29.453603 29455 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 1.
I20260809 07:31:29.456136 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:29.457162 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b56b8ed78214416a190e31e38e793ae" is_pre_election: true
I20260809 07:31:29.457937 29570 leader_election.cc:288] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:29.458747 29225 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.459309 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:29.460345 29338 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.465037 29284 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb; no voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:29.468566 29224 raft_consensus.cc:3037] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.468253 29340 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:29.469471 29554 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.488909 29554 raft_consensus.cc:2726] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:29.492780 29338 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:29.480484 29225 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 1.
I20260809 07:31:29.499739 29401 leader_election.cc:302] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:29.501634 29574 raft_consensus.cc:2781] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:29.505563 29574 raft_consensus.cc:686] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:29.506620 29574 consensus_queue.cc:237] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.508769 29070 catalog_manager.cc:5495] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:29.510272 29570 consensus_queue.cc:1035] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.510990 29574 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.523598 29570 consensus_queue.cc:1035] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.528825 29065 catalog_manager.cc:5495] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 1 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:29.549582 29574 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.575248 29069 catalog_manager.cc:5495] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 1 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:29.654552 29641 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.665771 29570 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:29.709884 29224 raft_consensus.cc:3037] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:29.717726 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:29.718475 29457 raft_consensus.cc:595] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Received request to transfer leadership to TS 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:29.727454 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:29.727936 29457 raft_consensus.cc:595] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Received request to transfer leadership to TS 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:29.749547 29224 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:35332
I20260809 07:31:29.750053 29224 raft_consensus.cc:595] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:29.755929 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "46df14febbe54742affebecfe152b06a"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:29.756635 29457 raft_consensus.cc:595] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:29.762640 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:29.763303 29457 raft_consensus.cc:595] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:29.770087 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:29.770815 29457 raft_consensus.cc:595] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:29.827539 29572 consensus_queue.cc:1035] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:29.842103 29572 consensus_queue.cc:1035] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.969447 29641 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:29.971690 29574 raft_consensus.cc:979] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:29.972222 29574 raft_consensus.cc:1067] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:29.974110 29340 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36162
I20260809 07:31:29.974884 29340 raft_consensus.cc:482] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:29.975381 29340 raft_consensus.cc:3037] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:29.991021 29340 raft_consensus.cc:504] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:29.992790 29575 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:30.007875 29575 raft_consensus.cc:979] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:30.008466 29575 raft_consensus.cc:1067] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Signalling peer 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:30.011951 29224 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:35312
I20260809 07:31:30.012745 29224 raft_consensus.cc:482] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:30.012554 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:30.013430 29225 raft_consensus.cc:3037] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.018486 29224 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.024701 29225 raft_consensus.cc:2454] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.026077 29280 leader_election.cc:302] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:30.027760 29554 raft_consensus.cc:2781] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:30.028090 29224 raft_consensus.cc:504] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.031488 29224 leader_election.cc:288] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 2 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:30.032459 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:30.033227 29338 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.034261 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:30.034476 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bbdfafc1344a22a8e1acf1da4b2846" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:30.035079 29454 raft_consensus.cc:3032] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:30.035283 29457 raft_consensus.cc:3032] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:30.035554 29454 raft_consensus.cc:729] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:30.035820 29457 raft_consensus.cc:729] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:30.037034 29454 consensus_queue.cc:260] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.038808 29454 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.037076 29457 consensus_queue.cc:260] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.051131 29457 raft_consensus.cc:3037] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.048026 29340 leader_election.cc:288] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:30.053925 29338 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 2.
I20260809 07:31:30.068863 29575 consensus_queue.cc:1035] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:30.071300 29631 raft_consensus.cc:979] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:30.071940 29631 raft_consensus.cc:1067] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:30.072268 29457 raft_consensus.cc:2454] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.073436 29454 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 2.
I20260809 07:31:30.077587 29166 leader_election.cc:302] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:30.078936 29529 raft_consensus.cc:2781] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:30.079648 29529 raft_consensus.cc:686] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:30.081063 29529 consensus_queue.cc:237] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.087296 29554 raft_consensus.cc:686] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:30.090009 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36166
I20260809 07:31:30.090885 29574 raft_consensus.cc:979] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:30.091306 29338 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:30.091344 29574 raft_consensus.cc:1067] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Signalling peer 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:30.092159 29338 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.090303 29554 consensus_queue.cc:237] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.098801 29224 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:35312
I20260809 07:31:30.099784 29224 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:30.100320 29224 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.111198 29641 consensus_queue.cc:1035] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:30.110956 29224 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.115612 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:30.114841 29065 catalog_manager.cc:5495] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 1 to 2, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:30.116395 29337 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.121517 29224 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 2 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:30.123147 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:30.123965 29454 raft_consensus.cc:3032] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:30.124435 29454 raft_consensus.cc:729] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:30.125744 29454 consensus_queue.cc:260] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.127955 29454 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.138109 29454 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 2.
I20260809 07:31:30.140128 29168 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:30.146749 29529 raft_consensus.cc:2781] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:30.147508 29529 raft_consensus.cc:686] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:30.149444 29529 consensus_queue.cc:237] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.143154 29338 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.160334 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:30.161233 29224 raft_consensus.cc:3032] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:30.161718 29224 raft_consensus.cc:729] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:30.163255 29224 consensus_queue.cc:260] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.165028 29224 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.138306 29337 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 2.
I20260809 07:31:30.177172 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:30.177879 29454 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.145211 29069 catalog_manager.cc:5495] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 1 to 2, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 2 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:30.177196 29338 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:30.188354 29224 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.190047 29280 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:30.191392 29554 raft_consensus.cc:2781] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:30.192028 29554 raft_consensus.cc:686] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:30.193224 29554 consensus_queue.cc:237] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.210567 29454 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.216019 29574 raft_consensus.cc:979] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:30.216670 29574 raft_consensus.cc:1067] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:30.226086 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "46df14febbe54742affebecfe152b06a"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36162
I20260809 07:31:30.226907 29338 raft_consensus.cc:482] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:30.227454 29338 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.227556 29575 raft_consensus.cc:979] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:30.228328 29574 raft_consensus.cc:1067] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:30.230593 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36162
I20260809 07:31:30.231241 29337 raft_consensus.cc:482] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:30.231678 29337 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.240257 29337 raft_consensus.cc:504] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.241441 29338 raft_consensus.cc:504] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.244513 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:30.245136 29224 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.249986 29338 leader_election.cc:288] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:30.257308 29337 leader_election.cc:288] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:30.259348 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:30.260987 29225 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.261245 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:30.261837 29457 raft_consensus.cc:3032] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:30.262128 29457 raft_consensus.cc:729] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:30.262234 29224 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.262996 29457 consensus_queue.cc:260] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.264714 29280 leader_election.cc:302] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:30.265626 29554 raft_consensus.cc:2781] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:30.266204 29554 raft_consensus.cc:686] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:30.268327 29454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:30.270665 29454 raft_consensus.cc:3032] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:30.268740 29554 consensus_queue.cc:237] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.271173 29454 raft_consensus.cc:729] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:30.270462 29457 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.275043 29454 consensus_queue.cc:260] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.279212 29454 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:30.287635 29225 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.289072 29280 leader_election.cc:302] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:30.290082 29454 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.290414 29554 raft_consensus.cc:2781] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:30.291015 29554 raft_consensus.cc:686] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:30.291910 29554 consensus_queue.cc:237] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:30.299837 29069 catalog_manager.cc:5495] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 1 to 2, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:30.306514 29457 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:30.313663 29065 catalog_manager.cc:5495] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 1 to 2, leader changed from 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 2 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:30.316846 29065 catalog_manager.cc:5495] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 1 to 2, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 2 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:30.343920 29069 catalog_manager.cc:5495] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 1 to 2, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 2 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:30.516700 29457 raft_consensus.cc:1261] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.517931 29554 consensus_queue.cc:1035] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.524582 29225 raft_consensus.cc:1261] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.526011 29554 consensus_queue.cc:1035] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.580168 29457 raft_consensus.cc:1261] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.581300 29529 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.587942 29337 raft_consensus.cc:1261] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.589356 29631 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.612996 29457 raft_consensus.cc:1261] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.614074 29674 consensus_queue.cc:1035] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.622098 29225 raft_consensus.cc:1261] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.624248 29674 consensus_queue.cc:1035] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.639773 29337 raft_consensus.cc:1261] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.641047 29675 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.648991 29457 raft_consensus.cc:1261] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.653394 29529 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.671851 29225 raft_consensus.cc:1261] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.672952 29674 consensus_queue.cc:1035] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.680030 29454 raft_consensus.cc:1261] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.681499 29554 consensus_queue.cc:1035] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.869596 29454 raft_consensus.cc:1261] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.870930 29554 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:30.878878 29225 raft_consensus.cc:1261] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:30.880532 29665 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:32.800408 29021 auto_leader_rebalancer-test.cc:269] The leader distribution is
I20260809 07:31:32.801235 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:32.824807 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
mode: GRACEFUL
new_leader_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:36168
I20260809 07:31:32.825320 29337 raft_consensus.cc:595] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Received request to transfer leadership to TS 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:32.835949 29225 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
mode: GRACEFUL
new_leader_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:35346
I20260809 07:31:32.836405 29225 raft_consensus.cc:595] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Received request to transfer leadership to TS 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:32.837340 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 07:31:32.837728 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:32.874090 29700 raft_consensus.cc:979] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: : Instructing follower 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:32.874577 29679 raft_consensus.cc:1067] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Signalling peer 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:32.875926 29454 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:49108
I20260809 07:31:32.876425 29454 raft_consensus.cc:482] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:32.876693 29454 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:32.881309 29454 raft_consensus.cc:504] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:32.882865 29454 leader_election.cc:288] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 3 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:32.883644 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:32.883832 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:32.884131 29225 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:32.884359 29337 raft_consensus.cc:3032] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:32.884662 29337 raft_consensus.cc:729] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:32.885257 29337 consensus_queue.cc:260] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:32.886097 29337 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:32.888324 29225 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 3.
I20260809 07:31:32.889183 29397 leader_election.cc:302] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:32.889788 29691 raft_consensus.cc:2781] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:32.890233 29337 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 3.
I20260809 07:31:32.891032 29691 raft_consensus.cc:686] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:32.891744 29691 consensus_queue.cc:237] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:32.899729 29069 catalog_manager.cc:5495] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 2 to 3, leader changed from 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66) to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 3 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:33.255861 29690 raft_consensus.cc:979] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: : Instructing follower 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:33.256253 29712 raft_consensus.cc:1067] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Signalling peer 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:33.257604 29454 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:49102
I20260809 07:31:33.258066 29454 raft_consensus.cc:482] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:33.258359 29454 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:33.264031 29454 raft_consensus.cc:504] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:33.266173 29454 leader_election.cc:288] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 2 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:33.266830 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:33.267135 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:33.267431 29225 raft_consensus.cc:3032] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:33.267661 29337 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:33.267853 29225 raft_consensus.cc:729] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:33.268987 29225 consensus_queue.cc:260] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:33.270809 29225 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:33.277273 29337 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 2.
I20260809 07:31:33.278802 29401 leader_election.cc:302] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:33.279845 29691 raft_consensus.cc:2781] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:33.280332 29225 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 2.
I20260809 07:31:33.280480 29691 raft_consensus.cc:686] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:33.281889 29691 consensus_queue.cc:237] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:33.292970 29069 catalog_manager.cc:5495] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 1 to 2, leader changed from 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65) to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 2 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:33.421622 29225 raft_consensus.cc:1261] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:33.422801 29691 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:33.437428 29337 raft_consensus.cc:1261] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:33.441176 29720 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:31:33.678458 29337 raft_consensus.cc:1261] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:33.680184 29720 consensus_queue.cc:1035] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:33.693799 29225 raft_consensus.cc:1261] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:33.695879 29721 consensus_queue.cc:1035] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:33.838501 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:33.843590 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:33.843946 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:34.844740 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:34.849669 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:34.850034 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:35.850960 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:35.855847 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:35.856168 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:36.856844 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:36.861467 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:36.861799 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:37.862479 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:37.866807 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:37.867120 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:38.867769 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:38.872885 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:38.873347 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:39.874207 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:39.878852 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:39.879194 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:40.879961 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:40.885591 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:40.886099 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:41.886931 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:41.891263 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:41.891578 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:42.915541 29021 auto_leader_rebalancer-test.cc:288] The leader distribution is
I20260809 07:31:42.919911 29225 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:35332
I20260809 07:31:42.920593 29225 raft_consensus.cc:595] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:42.924916 29225 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "70db551eedde440193453f228c8b3d27"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:35332
I20260809 07:31:42.925369 29225 raft_consensus.cc:595] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:42.929327 29225 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:35332
I20260809 07:31:42.929755 29225 raft_consensus.cc:595] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:42.933836 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:42.934270 29457 raft_consensus.cc:595] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:42.944504 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
mode: GRACEFUL
new_leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:49112
I20260809 07:31:42.944957 29457 raft_consensus.cc:595] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Received request to transfer leadership to TS 015d82d3a9824f689d3b0a54aa41e5cb
I20260809 07:31:43.016074 29740 raft_consensus.cc:979] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.016624 29741 raft_consensus.cc:1067] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.018133 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36166
I20260809 07:31:43.018639 29337 raft_consensus.cc:482] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:43.018958 29337 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.023871 29337 raft_consensus.cc:504] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.025386 29337 leader_election.cc:288] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 3 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:43.026297 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:43.026450 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:43.026914 29225 raft_consensus.cc:3032] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:43.026973 29457 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.027187 29225 raft_consensus.cc:729] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:43.027822 29225 consensus_queue.cc:260] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.028795 29225 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.031473 29457 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 3.
I20260809 07:31:43.032507 29284 leader_election.cc:302] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:43.033268 29738 raft_consensus.cc:2781] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:43.033620 29225 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 3.
I20260809 07:31:43.034268 29738 raft_consensus.cc:686] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:43.035303 29738 consensus_queue.cc:237] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.041712 29069 catalog_manager.cc:5495] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 2 to 3, leader changed from 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 3 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:43.103185 29740 raft_consensus.cc:979] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.103593 29741 raft_consensus.cc:1067] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.104861 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36166
I20260809 07:31:43.105275 29337 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:43.105523 29337 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.109439 29337 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.110728 29337 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 3 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:43.111586 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:43.111680 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:43.112234 29225 raft_consensus.cc:3032] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:43.112238 29457 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.112638 29225 raft_consensus.cc:729] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:43.113366 29225 consensus_queue.cc:260] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.114264 29225 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.116804 29457 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 3.
I20260809 07:31:43.117654 29284 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:43.118302 29738 raft_consensus.cc:2781] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:43.118480 29225 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 3.
I20260809 07:31:43.118793 29738 raft_consensus.cc:686] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:43.119648 29738 consensus_queue.cc:237] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.125437 29069 catalog_manager.cc:5495] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 2 to 3, leader changed from 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 3 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:43.277136 29739 raft_consensus.cc:979] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.277642 29764 raft_consensus.cc:1067] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.279053 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36162
I20260809 07:31:43.279505 29337 raft_consensus.cc:482] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:43.279764 29337 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:43.283562 29337 raft_consensus.cc:504] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.284852 29337 leader_election.cc:288] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 4 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:43.285631 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:43.285851 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a01215676044eb9b3d46bd768aa4ebb" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:43.286077 29225 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:43.286407 29457 raft_consensus.cc:3032] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:31:43.286662 29457 raft_consensus.cc:729] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:43.287307 29457 consensus_queue.cc:260] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.288313 29457 raft_consensus.cc:3037] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:43.289903 29225 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 4.
I20260809 07:31:43.290807 29280 leader_election.cc:302] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:43.291425 29738 raft_consensus.cc:2781] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:31:43.291913 29738 raft_consensus.cc:686] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:43.292871 29738 consensus_queue.cc:237] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.293502 29457 raft_consensus.cc:2454] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 4.
I20260809 07:31:43.299841 29069 catalog_manager.cc:5495] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 3 to 4, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 4 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:43.376133 29739 raft_consensus.cc:979] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.376579 29764 raft_consensus.cc:1067] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.377866 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36162
I20260809 07:31:43.378288 29337 raft_consensus.cc:482] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:43.378587 29337 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.382531 29337 raft_consensus.cc:504] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.383870 29337 leader_election.cc:288] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 3 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:43.384548 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:43.384815 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feaae344637446fbbd0330ecb39054f" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:43.385116 29225 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.385319 29457 raft_consensus.cc:3032] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:43.385629 29457 raft_consensus.cc:729] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:43.386269 29457 consensus_queue.cc:260] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.387166 29457 raft_consensus.cc:3037] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:43.389088 29225 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 3.
I20260809 07:31:43.389818 29280 leader_election.cc:302] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:43.390467 29738 raft_consensus.cc:2781] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:43.390915 29738 raft_consensus.cc:686] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:43.391793 29457 raft_consensus.cc:2454] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 3.
I20260809 07:31:43.391709 29738 consensus_queue.cc:237] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.399163 29069 catalog_manager.cc:5495] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 2 to 3, leader changed from 0b56b8ed78214416a190e31e38e793ae (127.28.87.67) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 3 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:43.469275 29740 raft_consensus.cc:979] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: : Instructing follower 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.469725 29741 raft_consensus.cc:1067] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Signalling peer 015d82d3a9824f689d3b0a54aa41e5cb to start an election
I20260809 07:31:43.471042 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "70db551eedde440193453f228c8b3d27"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
from {username='slave'} at 127.0.0.1:36166
I20260809 07:31:43.471482 29337 raft_consensus.cc:482] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:43.471698 29337 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:43.475714 29337 raft_consensus.cc:504] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.477066 29337 leader_election.cc:288] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:43.477794 29225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:43.478345 29225 raft_consensus.cc:3032] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:43.478261 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:43.478709 29225 raft_consensus.cc:729] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:43.478891 29457 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:43.479394 29225 consensus_queue.cc:260] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.480522 29225 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:43.481765 29454 raft_consensus.cc:1261] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:43.482784 29738 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:43.485152 29457 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:43.486129 29284 leader_election.cc:302] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015d82d3a9824f689d3b0a54aa41e5cb, 0b56b8ed78214416a190e31e38e793ae; no voters:
I20260809 07:31:43.486279 29225 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015d82d3a9824f689d3b0a54aa41e5cb in term 2.
I20260809 07:31:43.487165 29738 raft_consensus.cc:2781] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:43.487651 29738 raft_consensus.cc:686] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Leader. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:43.488509 29738 consensus_queue.cc:237] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:43.494827 29225 raft_consensus.cc:1261] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:43.498039 29746 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:31:43.507643 29069 catalog_manager.cc:5495] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb reported cstate change: term changed from 1 to 2, leader changed from 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65) to 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66). New cstate: current_term: 2 leader_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:43.524101 29454 raft_consensus.cc:1261] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:43.525086 29746 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:43.534179 29225 raft_consensus.cc:1261] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:43.535902 29746 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:43.875240 29225 raft_consensus.cc:1261] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:43.876392 29746 consensus_queue.cc:1035] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:43.884629 29454 raft_consensus.cc:1261] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:43.886425 29738 consensus_queue.cc:1035] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:43.908290 29454 raft_consensus.cc:1261] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:31:43.909283 29738 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:31:43.916939 29225 raft_consensus.cc:1261] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:31:43.920002 29738 consensus_queue.cc:1035] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:31:43.983752 29454 raft_consensus.cc:1261] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:43.984830 29738 consensus_queue.cc:1035] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:43.992414 29225 raft_consensus.cc:1261] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Refusing update from remote peer 015d82d3a9824f689d3b0a54aa41e5cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:43.994480 29738 consensus_queue.cc:1035] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:45.972187 29021 auto_leader_rebalancer-test.cc:303] The leader distribution is
I20260809 07:31:45.972898 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:45.979136 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "46df14febbe54742affebecfe152b06a"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
mode: GRACEFUL
new_leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:36168
I20260809 07:31:45.979797 29337 raft_consensus.cc:595] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Received request to transfer leadership to TS 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:45.982062 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "70db551eedde440193453f228c8b3d27"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
mode: GRACEFUL
new_leader_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:36168
I20260809 07:31:45.982494 29337 raft_consensus.cc:595] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Received request to transfer leadership to TS 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:45.984266 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
mode: GRACEFUL
new_leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:36168
I20260809 07:31:45.984659 29337 raft_consensus.cc:595] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Received request to transfer leadership to TS 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:45.986474 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
mode: GRACEFUL
new_leader_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:36168
I20260809 07:31:45.986842 29337 raft_consensus.cc:595] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Received request to transfer leadership to TS 0b56b8ed78214416a190e31e38e793ae
I20260809 07:31:45.988718 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305"
dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
mode: GRACEFUL
new_leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:36168
I20260809 07:31:45.989095 29337 raft_consensus.cc:595] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Received request to transfer leadership to TS 89ed39c1f88b4fb18cb9b5712dcaa026
I20260809 07:31:45.989979 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:31:45.990437 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:45.990839 29799 raft_consensus.cc:979] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: : Instructing follower 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:45.991336 29801 raft_consensus.cc:1067] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Signalling peer 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:45.992678 29224 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:35322
I20260809 07:31:45.993062 29224 raft_consensus.cc:482] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:45.993378 29224 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:45.998478 29224 raft_consensus.cc:504] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.000311 29224 leader_election.cc:288] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 3 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:46.001278 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:46.001453 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bcc74c7c04e57bcfe6a4f5e01c305" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:46.002046 29337 raft_consensus.cc:3032] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:46.002113 29457 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.002345 29337 raft_consensus.cc:729] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:46.003120 29337 consensus_queue.cc:260] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.004102 29337 raft_consensus.cc:3037] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.008083 29457 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 3.
I20260809 07:31:46.009099 29168 leader_election.cc:302] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:46.009470 29337 raft_consensus.cc:2454] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 3.
I20260809 07:31:46.010667 29802 raft_consensus.cc:2781] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:46.011824 29802 raft_consensus.cc:686] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:46.012645 29802 consensus_queue.cc:237] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.019023 29069 catalog_manager.cc:5495] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 2 to 3, leader changed from 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66) to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:46.097466 29799 raft_consensus.cc:979] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: : Instructing follower 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:46.097899 29801 raft_consensus.cc:1067] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Signalling peer 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:46.099258 29224 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:35322
I20260809 07:31:46.099653 29224 raft_consensus.cc:482] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:46.099870 29224 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:46.104094 29224 raft_consensus.cc:504] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.105435 29224 leader_election.cc:288] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 4 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:46.106155 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:46.106402 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dda79df745442e7a5c99fd7d7bb00a2" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:46.106638 29337 raft_consensus.cc:3032] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:31:46.106949 29337 raft_consensus.cc:729] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:46.106957 29457 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:46.107651 29337 consensus_queue.cc:260] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.108741 29337 raft_consensus.cc:3037] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:46.114537 29457 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 4.
I20260809 07:31:46.115095 29337 raft_consensus.cc:2454] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 4.
I20260809 07:31:46.115660 29168 leader_election.cc:302] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:46.116446 29802 raft_consensus.cc:2781] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:31:46.116940 29802 raft_consensus.cc:686] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:46.117871 29802 consensus_queue.cc:237] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.118626 29801 raft_consensus.cc:979] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: : Instructing follower 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:46.119273 29816 raft_consensus.cc:1067] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Signalling peer 89ed39c1f88b4fb18cb9b5712dcaa026 to start an election
I20260809 07:31:46.121235 29224 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "46df14febbe54742affebecfe152b06a"
dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
from {username='slave'} at 127.0.0.1:35322
I20260809 07:31:46.121663 29224 raft_consensus.cc:482] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:46.123471 29224 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.125396 29069 catalog_manager.cc:5495] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 3 to 4, leader changed from 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66) to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:46.127763 29224 raft_consensus.cc:504] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.129470 29224 leader_election.cc:288] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 3 election: Requested vote from peers 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527), 0b56b8ed78214416a190e31e38e793ae (127.28.87.67:46033)
I20260809 07:31:46.130225 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:46.130404 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46df14febbe54742affebecfe152b06a" candidate_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
I20260809 07:31:46.130777 29337 raft_consensus.cc:3032] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:46.130857 29457 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.131093 29337 raft_consensus.cc:729] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:46.131747 29337 consensus_queue.cc:260] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.132812 29337 raft_consensus.cc:3037] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.135735 29457 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 3.
I20260809 07:31:46.136524 29168 leader_election.cc:302] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:46.137133 29802 raft_consensus.cc:2781] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:46.137526 29337 raft_consensus.cc:2454] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ed39c1f88b4fb18cb9b5712dcaa026 in term 3.
I20260809 07:31:46.137688 29802 raft_consensus.cc:686] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 LEADER]: Becoming Leader. State: Replica: 89ed39c1f88b4fb18cb9b5712dcaa026, State: Running, Role: LEADER
I20260809 07:31:46.138612 29802 consensus_queue.cc:237] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.145362 29069 catalog_manager.cc:5495] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 reported cstate change: term changed from 2 to 3, leader changed from 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66) to 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:46.180387 29816 raft_consensus.cc:979] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: : Instructing follower 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:46.180902 29817 raft_consensus.cc:1067] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Signalling peer 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:46.182482 29457 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "70db551eedde440193453f228c8b3d27"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:49108
I20260809 07:31:46.183028 29457 raft_consensus.cc:482] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:46.183321 29457 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.187482 29457 raft_consensus.cc:504] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.189198 29457 leader_election.cc:288] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 3 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:46.189998 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:46.190145 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70db551eedde440193453f228c8b3d27" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:46.190577 29224 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.190620 29337 raft_consensus.cc:3032] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:31:46.191013 29337 raft_consensus.cc:729] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:46.191732 29337 consensus_queue.cc:260] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.192883 29337 raft_consensus.cc:3037] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:31:46.196375 29224 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 3.
I20260809 07:31:46.197484 29397 leader_election.cc:302] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:46.198218 29764 raft_consensus.cc:2781] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:31:46.198263 29337 raft_consensus.cc:2454] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 3.
I20260809 07:31:46.199579 29764 raft_consensus.cc:686] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:46.200402 29764 consensus_queue.cc:237] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.206655 29069 catalog_manager.cc:5495] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 2 to 3, leader changed from 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66) to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 3 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:46.230867 29816 raft_consensus.cc:979] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: : Instructing follower 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:46.231328 29817 raft_consensus.cc:1067] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Signalling peer 0b56b8ed78214416a190e31e38e793ae to start an election
I20260809 07:31:46.232545 29457 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc"
dest_uuid: "0b56b8ed78214416a190e31e38e793ae"
from {username='slave'} at 127.0.0.1:49108
I20260809 07:31:46.232968 29457 raft_consensus.cc:482] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:46.233207 29457 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:46.237088 29457 raft_consensus.cc:504] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.238337 29457 leader_election.cc:288] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 4 election: Requested vote from peers 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199), 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527)
I20260809 07:31:46.239028 29224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026"
I20260809 07:31:46.239246 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93670b2e26c7405ca3bc86729c953acc" candidate_uuid: "0b56b8ed78214416a190e31e38e793ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "015d82d3a9824f689d3b0a54aa41e5cb"
I20260809 07:31:46.239563 29224 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:46.239717 29337 raft_consensus.cc:3032] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:31:46.240016 29337 raft_consensus.cc:729] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015d82d3a9824f689d3b0a54aa41e5cb, State: Running, Role: LEADER
I20260809 07:31:46.240669 29337 consensus_queue.cc:260] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.241797 29337 raft_consensus.cc:3037] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:31:46.245398 29224 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 4.
I20260809 07:31:46.246301 29397 leader_election.cc:302] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b56b8ed78214416a190e31e38e793ae, 89ed39c1f88b4fb18cb9b5712dcaa026; no voters:
I20260809 07:31:46.247000 29764 raft_consensus.cc:2781] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:31:46.247321 29337 raft_consensus.cc:2454] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b56b8ed78214416a190e31e38e793ae in term 4.
I20260809 07:31:46.247474 29764 raft_consensus.cc:686] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 4 LEADER]: Becoming Leader. State: Replica: 0b56b8ed78214416a190e31e38e793ae, State: Running, Role: LEADER
I20260809 07:31:46.248379 29764 consensus_queue.cc:237] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } }
I20260809 07:31:46.254693 29069 catalog_manager.cc:5495] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae reported cstate change: term changed from 3 to 4, leader changed from 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66) to 0b56b8ed78214416a190e31e38e793ae (127.28.87.67). New cstate: current_term: 4 leader_uuid: "0b56b8ed78214416a190e31e38e793ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:46.422493 29337 raft_consensus.cc:1261] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:46.423585 29802 consensus_queue.cc:1035] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:46.432989 29457 raft_consensus.cc:1261] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:46.434517 29828 consensus_queue.cc:1035] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:31:46.503432 29457 raft_consensus.cc:1261] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:31:46.504489 29828 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:31:46.514477 29337 raft_consensus.cc:1261] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:31:46.516309 29802 consensus_queue.cc:1035] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:31:46.596774 29337 raft_consensus.cc:1261] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:46.597888 29802 consensus_queue.cc:1035] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:46.607522 29457 raft_consensus.cc:1261] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Refusing update from remote peer 89ed39c1f88b4fb18cb9b5712dcaa026: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:46.608932 29803 consensus_queue.cc:1035] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b56b8ed78214416a190e31e38e793ae" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:31:46.678808 29224 raft_consensus.cc:1261] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:46.679879 29764 consensus_queue.cc:1035] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:46.688403 29337 raft_consensus.cc:1261] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:31:46.689764 29839 consensus_queue.cc:1035] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:31:46.801966 29224 raft_consensus.cc:1261] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:31:46.803187 29839 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ed39c1f88b4fb18cb9b5712dcaa026" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 41199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:31:46.811414 29337 raft_consensus.cc:1261] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Refusing update from remote peer 0b56b8ed78214416a190e31e38e793ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:31:46.813160 29839 consensus_queue.cc:1035] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "015d82d3a9824f689d3b0a54aa41e5cb" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:31:46.991247 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:46.996470 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:46.996793 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:47.997668 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:48.002404 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:48.002794 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:49.003532 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:49.008021 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:49.008396 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:50.009130 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:50.013659 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:50.014021 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:51.014868 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:51.019598 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:51.020027 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:52.020867 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:52.026125 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:52.026494 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:53.027242 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:53.032476 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:53.032841 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:54.033608 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:54.038496 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:54.038857 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:55.039703 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:31:55.044384 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:31:55.044715 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:31:56.071470 29021 auto_leader_rebalancer-test.cc:314] The leader distribution is
I20260809 07:31:56.072088 29021 tablet_server.cc:178] TabletServer@127.28.87.65:0 shutting down...
W20260809 07:31:56.079859 29280 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260809 07:31:56.084065 29280 consensus_peers.cc:487] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb -> Peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Couldn't send request to peer 89ed39c1f88b4fb18cb9b5712dcaa026. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260809 07:31:56.102567 29397 consensus_peers.cc:487] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae -> Peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Couldn't send request to peer 89ed39c1f88b4fb18cb9b5712dcaa026. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:31:56.103473 29021 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 07:31:56.104143 29021 tablet_replica.cc:330] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.104857 29021 raft_consensus.cc:2229] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.105448 29021 raft_consensus.cc:2258] T 4c336377b4204dca9c20db902c911a95 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.107968 29021 tablet_replica.cc:330] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.108654 29021 raft_consensus.cc:2229] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.109131 29021 raft_consensus.cc:2258] T 21bbdfafc1344a22a8e1acf1da4b2846 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.111075 29021 tablet_replica.cc:330] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.111547 29021 raft_consensus.cc:2229] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.111924 29021 raft_consensus.cc:2258] T 8feaae344637446fbbd0330ecb39054f P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.113420 29021 tablet_replica.cc:330] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.113852 29021 raft_consensus.cc:2229] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.114692 29021 raft_consensus.cc:2258] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.116254 29021 tablet_replica.cc:330] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.116686 29021 raft_consensus.cc:2229] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.117415 29021 raft_consensus.cc:2258] T 7dda79df745442e7a5c99fd7d7bb00a2 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.118894 29021 tablet_replica.cc:330] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.119311 29021 raft_consensus.cc:2229] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.119942 29021 raft_consensus.cc:2258] T 46df14febbe54742affebecfe152b06a P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.121374 29021 tablet_replica.cc:330] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.121789 29021 raft_consensus.cc:2229] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.122218 29021 raft_consensus.cc:2258] T 5a01215676044eb9b3d46bd768aa4ebb P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.123574 29021 tablet_replica.cc:330] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.123987 29021 raft_consensus.cc:2229] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.124382 29021 raft_consensus.cc:2258] T 70db551eedde440193453f228c8b3d27 P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.125834 29021 tablet_replica.cc:330] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026: stopping tablet replica
I20260809 07:31:56.126219 29021 raft_consensus.cc:2229] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.126638 29021 raft_consensus.cc:2258] T 93670b2e26c7405ca3bc86729c953acc P 89ed39c1f88b4fb18cb9b5712dcaa026 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260809 07:31:56.208052 29280 consensus_peers.cc:487] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb -> Peer 89ed39c1f88b4fb18cb9b5712dcaa026 (127.28.87.65:41199): Couldn't send request to peer 89ed39c1f88b4fb18cb9b5712dcaa026. Status: Network error: Client connection negotiation failed: client connection to 127.28.87.65:41199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:31:56.208696 29021 tablet_server.cc:194] TabletServer@127.28.87.65:0 shutdown complete.
I20260809 07:31:56.226954 29021 tablet_server.cc:178] TabletServer@127.28.87.66:0 shutting down...
I20260809 07:31:56.250993 29021 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 07:31:56.251643 29021 tablet_replica.cc:330] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.252584 29021 raft_consensus.cc:2229] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.253238 29021 raft_consensus.cc:2258] T 7dda79df745442e7a5c99fd7d7bb00a2 P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.254922 29021 tablet_replica.cc:330] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.255394 29021 raft_consensus.cc:2229] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.255759 29021 raft_consensus.cc:2258] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.257228 29021 tablet_replica.cc:330] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.257639 29021 raft_consensus.cc:2229] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.258419 29021 raft_consensus.cc:2258] T 8feaae344637446fbbd0330ecb39054f P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.259920 29021 tablet_replica.cc:330] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.260362 29021 raft_consensus.cc:2229] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.261096 29021 raft_consensus.cc:2258] T 21bbdfafc1344a22a8e1acf1da4b2846 P 015d82d3a9824f689d3b0a54aa41e5cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.262629 29021 tablet_replica.cc:330] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.263077 29021 raft_consensus.cc:2229] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.263736 29021 raft_consensus.cc:2258] T 5a01215676044eb9b3d46bd768aa4ebb P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.265213 29021 tablet_replica.cc:330] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.265614 29021 raft_consensus.cc:2229] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.265947 29021 raft_consensus.cc:2258] T 46df14febbe54742affebecfe152b06a P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.267478 29021 tablet_replica.cc:330] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.267913 29021 raft_consensus.cc:2229] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.268230 29021 raft_consensus.cc:2258] T 4c336377b4204dca9c20db902c911a95 P 015d82d3a9824f689d3b0a54aa41e5cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.269786 29021 tablet_replica.cc:330] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.270179 29021 raft_consensus.cc:2229] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.270576 29021 raft_consensus.cc:2258] T 93670b2e26c7405ca3bc86729c953acc P 015d82d3a9824f689d3b0a54aa41e5cb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.272109 29021 tablet_replica.cc:330] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb: stopping tablet replica
I20260809 07:31:56.272529 29021 raft_consensus.cc:2229] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.272928 29021 raft_consensus.cc:2258] T 70db551eedde440193453f228c8b3d27 P 015d82d3a9824f689d3b0a54aa41e5cb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260809 07:31:56.275393 29401 consensus_peers.cc:487] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae -> Peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Couldn't send request to peer 015d82d3a9824f689d3b0a54aa41e5cb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:31:56.299950 29021 tablet_server.cc:194] TabletServer@127.28.87.66:0 shutdown complete.
W20260809 07:31:56.312493 29401 consensus_peers.cc:487] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae -> Peer 015d82d3a9824f689d3b0a54aa41e5cb (127.28.87.66:37527): Couldn't send request to peer 015d82d3a9824f689d3b0a54aa41e5cb. Status: Network error: Client connection negotiation failed: client connection to 127.28.87.66:37527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:31:56.318143 29021 tablet_server.cc:178] TabletServer@127.28.87.67:0 shutting down...
I20260809 07:31:56.343242 29021 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 07:31:56.344081 29021 tablet_replica.cc:330] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.344923 29021 raft_consensus.cc:2229] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.345569 29021 raft_consensus.cc:2258] T 46df14febbe54742affebecfe152b06a P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.348240 29021 tablet_replica.cc:330] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.348951 29021 raft_consensus.cc:2229] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.349488 29021 raft_consensus.cc:2258] T 5a01215676044eb9b3d46bd768aa4ebb P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.351647 29021 tablet_replica.cc:330] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.352306 29021 raft_consensus.cc:2229] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 4 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.353323 29021 raft_consensus.cc:2258] T 93670b2e26c7405ca3bc86729c953acc P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.355021 29021 tablet_replica.cc:330] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.355437 29021 raft_consensus.cc:2229] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.356292 29021 raft_consensus.cc:2258] T 70db551eedde440193453f228c8b3d27 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.357796 29021 tablet_replica.cc:330] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.358263 29021 raft_consensus.cc:2229] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.358650 29021 raft_consensus.cc:2258] T 8feaae344637446fbbd0330ecb39054f P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.360242 29021 tablet_replica.cc:330] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.360700 29021 raft_consensus.cc:2229] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.361056 29021 raft_consensus.cc:2258] T 21bbdfafc1344a22a8e1acf1da4b2846 P 0b56b8ed78214416a190e31e38e793ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.362499 29021 tablet_replica.cc:330] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.362888 29021 raft_consensus.cc:2229] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.363545 29021 raft_consensus.cc:2258] T 4c336377b4204dca9c20db902c911a95 P 0b56b8ed78214416a190e31e38e793ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.364948 29021 tablet_replica.cc:330] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.365338 29021 raft_consensus.cc:2229] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.365733 29021 raft_consensus.cc:2258] T 7dda79df745442e7a5c99fd7d7bb00a2 P 0b56b8ed78214416a190e31e38e793ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.367161 29021 tablet_replica.cc:330] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae: stopping tablet replica
I20260809 07:31:56.367537 29021 raft_consensus.cc:2229] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:31:56.367870 29021 raft_consensus.cc:2258] T ff6bcc74c7c04e57bcfe6a4f5e01c305 P 0b56b8ed78214416a190e31e38e793ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.393819 29021 tablet_server.cc:194] TabletServer@127.28.87.67:0 shutdown complete.
I20260809 07:31:56.410607 29021 master.cc:537] Master@127.28.87.126:36317 shutting down...
I20260809 07:31:56.773157 29021 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:31:56.773793 29021 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:31:56.774205 29021 tablet_replica.cc:330] T 00000000000000000000000000000000 P f0b527a1ae9446ba9d3df3edfbb772bc: stopping tablet replica
I20260809 07:31:56.792711 29021 master.cc:552] Master@127.28.87.126:36317 shutdown complete.
015d82d3a9824f689d3b0a54aa41e5cb 4
0b56b8ed78214416a190e31e38e793ae 1
89ed39c1f88b4fb18cb9b5712dcaa026 4
015d82d3a9824f689d3b0a54aa41e5cb 3
0b56b8ed78214416a190e31e38e793ae 3
89ed39c1f88b4fb18cb9b5712dcaa026 3
015d82d3a9824f689d3b0a54aa41e5cb 8
0b56b8ed78214416a190e31e38e793ae 1
89ed39c1f88b4fb18cb9b5712dcaa026 0
015d82d3a9824f689d3b0a54aa41e5cb 3
0b56b8ed78214416a190e31e38e793ae 3
89ed39c1f88b4fb18cb9b5712dcaa026 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (31308 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260809 07:31:56.831048 29021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.28.87.126:39357
I20260809 07:31:56.832108 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:56.837834 29866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:56.837906 29867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:56.838681 29871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:56.839813 29021 server_base.cc:1034] running on GCE node
I20260809 07:31:56.840894 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:56.841066 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:56.841171 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260716841163 us; error 0 us; skew 500 ppm
I20260809 07:31:56.841617 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:56.847102 29021 webserver.cc:460] Webserver started at http://127.28.87.126:40339/ using document root <none> and password file <none>
I20260809 07:31:56.847551 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:56.847702 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:56.847939 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:56.849041 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/master-0-root/instance:
uuid: "ad53a97a568041579292455f0df871f3"
format_stamp: "Formatted at 2026-08-09 07:31:56 on dist-test-slave-fgb3"
I20260809 07:31:56.853466 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:31:56.856632 29881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:56.857311 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:31:56.857566 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/master-0-root
uuid: "ad53a97a568041579292455f0df871f3"
format_stamp: "Formatted at 2026-08-09 07:31:56 on dist-test-slave-fgb3"
I20260809 07:31:56.857832 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:56.883296 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:56.884577 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:56.921430 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.126:39357
I20260809 07:31:56.921515 29971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.126:39357 every 8 connection(s)
I20260809 07:31:56.925253 29972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:56.934486 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3: Bootstrap starting.
I20260809 07:31:56.938732 29972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:56.942713 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3: No bootstrap required, opened a new log
I20260809 07:31:56.944657 29972 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad53a97a568041579292455f0df871f3" member_type: VOTER }
I20260809 07:31:56.945111 29972 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:56.945333 29972 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad53a97a568041579292455f0df871f3, State: Initialized, Role: FOLLOWER
I20260809 07:31:56.945874 29972 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad53a97a568041579292455f0df871f3" member_type: VOTER }
I20260809 07:31:56.946364 29972 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:31:56.946610 29972 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:31:56.946871 29972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:56.950569 29972 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad53a97a568041579292455f0df871f3" member_type: VOTER }
I20260809 07:31:56.951043 29972 leader_election.cc:302] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad53a97a568041579292455f0df871f3; no voters:
I20260809 07:31:56.952108 29972 leader_election.cc:288] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:31:56.952440 29977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:56.953593 29977 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 1 LEADER]: Becoming Leader. State: Replica: ad53a97a568041579292455f0df871f3, State: Running, Role: LEADER
I20260809 07:31:56.954233 29977 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad53a97a568041579292455f0df871f3" member_type: VOTER }
I20260809 07:31:56.954828 29972 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:31:56.958806 29978 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad53a97a568041579292455f0df871f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad53a97a568041579292455f0df871f3" member_type: VOTER } }
I20260809 07:31:56.960430 29978 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [sys.catalog]: This master's current role is: LEADER
I20260809 07:31:56.959828 29979 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad53a97a568041579292455f0df871f3. Latest consensus state: current_term: 1 leader_uuid: "ad53a97a568041579292455f0df871f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad53a97a568041579292455f0df871f3" member_type: VOTER } }
I20260809 07:31:56.961565 29979 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [sys.catalog]: This master's current role is: LEADER
I20260809 07:31:56.962405 29990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:31:56.967028 29990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:31:56.968973 29021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260809 07:31:56.974905 29990 catalog_manager.cc:1347] Generated new cluster ID: 5e1f79692b8744d0ae233a213474d1ac
I20260809 07:31:56.975154 29990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:31:56.987573 29990 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:31:56.989064 29990 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:31:57.001821 29990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3: Generated new TSK 0
I20260809 07:31:57.002524 29990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:31:57.036021 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:57.041926 30013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:57.042762 30014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:57.044473 30016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:57.044804 29021 server_base.cc:1034] running on GCE node
I20260809 07:31:57.045651 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:57.045857 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:57.045996 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260717045981 us; error 0 us; skew 500 ppm
I20260809 07:31:57.046535 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:57.048722 29021 webserver.cc:460] Webserver started at http://127.28.87.65:46089/ using document root <none> and password file <none>
I20260809 07:31:57.049201 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:57.049379 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:57.049621 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:57.050714 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-0-root/instance:
uuid: "46a203ae94664aadaa6699d61543aec3"
format_stamp: "Formatted at 2026-08-09 07:31:57 on dist-test-slave-fgb3"
I20260809 07:31:57.054951 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:31:57.057969 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.058734 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:31:57.058979 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-0-root
uuid: "46a203ae94664aadaa6699d61543aec3"
format_stamp: "Formatted at 2026-08-09 07:31:57 on dist-test-slave-fgb3"
I20260809 07:31:57.059244 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:57.085554 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:57.086817 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:57.088362 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:31:57.090598 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:31:57.090795 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.091024 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:31:57.091180 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.129323 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.65:37055
I20260809 07:31:57.129467 30148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.65:37055 every 8 connection(s)
I20260809 07:31:57.133872 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:57.141084 30154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:57.141291 30158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:57.144197 30166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:57.144965 29021 server_base.cc:1034] running on GCE node
I20260809 07:31:57.146582 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:31:57.146898 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:57.147099 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260717147082 us; error 0 us; skew 500 ppm
I20260809 07:31:57.147634 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:57.148056 30149 heartbeater.cc:347] Connected to a master server at 127.28.87.126:39357
I20260809 07:31:57.148450 30149 heartbeater.cc:464] Registering TS with master...
I20260809 07:31:57.149236 30149 heartbeater.cc:511] Master 127.28.87.126:39357 requested a full tablet report, sending...
I20260809 07:31:57.150159 29021 webserver.cc:460] Webserver started at http://127.28.87.66:42541/ using document root <none> and password file <none>
I20260809 07:31:57.150722 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:57.150918 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:57.151160 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:57.151234 29912 ts_manager.cc:194] Registered new tserver with Master: 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055)
I20260809 07:31:57.152252 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-1-root/instance:
uuid: "3987da397aeb4c19982f3328ea51314f"
format_stamp: "Formatted at 2026-08-09 07:31:57 on dist-test-slave-fgb3"
I20260809 07:31:57.153057 29912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59426
I20260809 07:31:57.156858 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:31:57.160286 30171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.160980 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:31:57.161229 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-1-root
uuid: "3987da397aeb4c19982f3328ea51314f"
format_stamp: "Formatted at 2026-08-09 07:31:57 on dist-test-slave-fgb3"
I20260809 07:31:57.161482 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:57.182090 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:57.183223 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:57.184602 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:31:57.186718 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:31:57.186910 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.187122 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:31:57.187264 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.224452 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.66:36843
I20260809 07:31:57.224561 30270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.66:36843 every 8 connection(s)
I20260809 07:31:57.229308 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:31:57.235736 30277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:31:57.237921 30278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:57.240909 29021 server_base.cc:1034] running on GCE node
W20260809 07:31:57.241707 30281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:31:57.242609 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260809 07:31:57.242712 30271 heartbeater.cc:347] Connected to a master server at 127.28.87.126:39357
W20260809 07:31:57.242882 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:31:57.243091 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260717243076 us; error 0 us; skew 500 ppm
I20260809 07:31:57.243104 30271 heartbeater.cc:464] Registering TS with master...
I20260809 07:31:57.243659 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:31:57.243855 30271 heartbeater.cc:511] Master 127.28.87.126:39357 requested a full tablet report, sending...
I20260809 07:31:57.245688 29912 ts_manager.cc:194] Registered new tserver with Master: 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:57.246246 29021 webserver.cc:460] Webserver started at http://127.28.87.67:38183/ using document root <none> and password file <none>
I20260809 07:31:57.246770 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:31:57.246950 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:31:57.247185 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:31:57.247345 29912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59438
I20260809 07:31:57.248493 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-2-root/instance:
uuid: "a421085add2a4883856e420b4d38db00"
format_stamp: "Formatted at 2026-08-09 07:31:57 on dist-test-slave-fgb3"
I20260809 07:31:57.252671 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 07:31:57.255817 30286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.256558 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:31:57.256822 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-2-root
uuid: "a421085add2a4883856e420b4d38db00"
format_stamp: "Formatted at 2026-08-09 07:31:57 on dist-test-slave-fgb3"
I20260809 07:31:57.257076 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786260685288232-29021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:31:57.277100 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:31:57.278240 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:31:57.279665 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:31:57.281738 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:31:57.281929 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.282140 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:31:57.282279 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:31:57.324155 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.67:39327
I20260809 07:31:57.324245 30391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.67:39327 every 8 connection(s)
I20260809 07:31:57.337991 30392 heartbeater.cc:347] Connected to a master server at 127.28.87.126:39357
I20260809 07:31:57.338370 30392 heartbeater.cc:464] Registering TS with master...
I20260809 07:31:57.339051 30392 heartbeater.cc:511] Master 127.28.87.126:39357 requested a full tablet report, sending...
I20260809 07:31:57.340777 29912 ts_manager.cc:194] Registered new tserver with Master: a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.341595 29021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014396664s
I20260809 07:31:57.342232 29912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59446
I20260809 07:31:57.343973 29021 test_util.cc:268] Using random seed: 998806737
I20260809 07:31:57.367484 29912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59456:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:31:57.370283 29912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:31:57.455837 30327 tablet_service.cc:1467] Processing CreateTablet for tablet 09825d9aebf344c1904f7bed4e5cfaaf (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260809 07:31:57.457350 30327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09825d9aebf344c1904f7bed4e5cfaaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.457510 30319 tablet_service.cc:1467] Processing CreateTablet for tablet abe2396b43db427c9f4bcaabca4d7589 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260809 07:31:57.458632 30322 tablet_service.cc:1467] Processing CreateTablet for tablet d9cfc16f085e4df6a9b8b0c640716591 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260809 07:31:57.458892 30326 tablet_service.cc:1467] Processing CreateTablet for tablet e362b890f4884247b19c99202237dfc2 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260809 07:31:57.459199 30316 tablet_service.cc:1467] Processing CreateTablet for tablet d773aaa943354683b0b3439ccb5bba10 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260809 07:31:57.460618 30325 tablet_service.cc:1467] Processing CreateTablet for tablet 8f5d2edb16624218b5f8229e63385b4d (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260809 07:31:57.460794 30315 tablet_service.cc:1467] Processing CreateTablet for tablet 793bb066bf22463e9bd510ab0984ddce (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260809 07:31:57.462721 30312 tablet_service.cc:1467] Processing CreateTablet for tablet 513a5ae4a1754e57bc8d200f326a00ac (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260809 07:31:57.463797 30313 tablet_service.cc:1467] Processing CreateTablet for tablet f98ba7543fca47bba392925a1ef848c0 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260809 07:31:57.460669 30320 tablet_service.cc:1467] Processing CreateTablet for tablet 213881c3cf094e188ad18873996bc958 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260809 07:31:57.464823 30319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe2396b43db427c9f4bcaabca4d7589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.466894 30320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213881c3cf094e188ad18873996bc958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.467470 30312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513a5ae4a1754e57bc8d200f326a00ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.468026 30316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d773aaa943354683b0b3439ccb5bba10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.468648 30313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98ba7543fca47bba392925a1ef848c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.468643 30078 tablet_service.cc:1467] Processing CreateTablet for tablet 09825d9aebf344c1904f7bed4e5cfaaf (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260809 07:31:57.469018 30072 tablet_service.cc:1467] Processing CreateTablet for tablet e362b890f4884247b19c99202237dfc2 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260809 07:31:57.469595 30315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793bb066bf22463e9bd510ab0984ddce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.469909 30078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09825d9aebf344c1904f7bed4e5cfaaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.470170 30326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e362b890f4884247b19c99202237dfc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.471428 30071 tablet_service.cc:1467] Processing CreateTablet for tablet 8f5d2edb16624218b5f8229e63385b4d (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260809 07:31:57.471997 30072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e362b890f4884247b19c99202237dfc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.472616 30071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f5d2edb16624218b5f8229e63385b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.473172 30322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9cfc16f085e4df6a9b8b0c640716591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.475556 30325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f5d2edb16624218b5f8229e63385b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.480322 30070 tablet_service.cc:1467] Processing CreateTablet for tablet d9cfc16f085e4df6a9b8b0c640716591 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260809 07:31:57.481524 30070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9cfc16f085e4df6a9b8b0c640716591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.483911 30068 tablet_service.cc:1467] Processing CreateTablet for tablet 213881c3cf094e188ad18873996bc958 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260809 07:31:57.485148 30068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213881c3cf094e188ad18873996bc958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.487468 30067 tablet_service.cc:1467] Processing CreateTablet for tablet abe2396b43db427c9f4bcaabca4d7589 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260809 07:31:57.488693 30067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe2396b43db427c9f4bcaabca4d7589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.490893 30415 tablet_bootstrap.cc:492] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.491510 30066 tablet_service.cc:1467] Processing CreateTablet for tablet d773aaa943354683b0b3439ccb5bba10 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260809 07:31:57.492722 30066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d773aaa943354683b0b3439ccb5bba10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.495458 30065 tablet_service.cc:1467] Processing CreateTablet for tablet 793bb066bf22463e9bd510ab0984ddce (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260809 07:31:57.496721 30065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793bb066bf22463e9bd510ab0984ddce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.496650 30415 tablet_bootstrap.cc:654] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.499363 30078 tablet_service.cc:1467] Processing CreateTablet for tablet 513a5ae4a1754e57bc8d200f326a00ac (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260809 07:31:57.499469 30064 tablet_service.cc:1467] Processing CreateTablet for tablet f98ba7543fca47bba392925a1ef848c0 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260809 07:31:57.500564 30078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513a5ae4a1754e57bc8d200f326a00ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.503757 30064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98ba7543fca47bba392925a1ef848c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.512100 30209 tablet_service.cc:1467] Processing CreateTablet for tablet 8f5d2edb16624218b5f8229e63385b4d (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260809 07:31:57.513446 30209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f5d2edb16624218b5f8229e63385b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.516083 30210 tablet_service.cc:1467] Processing CreateTablet for tablet e362b890f4884247b19c99202237dfc2 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260809 07:31:57.514597 30211 tablet_service.cc:1467] Processing CreateTablet for tablet 09825d9aebf344c1904f7bed4e5cfaaf (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260809 07:31:57.517098 30199 tablet_service.cc:1467] Processing CreateTablet for tablet 513a5ae4a1754e57bc8d200f326a00ac (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260809 07:31:57.521628 30203 tablet_service.cc:1467] Processing CreateTablet for tablet 793bb066bf22463e9bd510ab0984ddce (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260809 07:31:57.523109 30200 tablet_service.cc:1467] Processing CreateTablet for tablet f98ba7543fca47bba392925a1ef848c0 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260809 07:31:57.526136 30206 tablet_service.cc:1467] Processing CreateTablet for tablet abe2396b43db427c9f4bcaabca4d7589 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260809 07:31:57.527726 30207 tablet_service.cc:1467] Processing CreateTablet for tablet 213881c3cf094e188ad18873996bc958 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260809 07:31:57.529141 30205 tablet_service.cc:1467] Processing CreateTablet for tablet d773aaa943354683b0b3439ccb5bba10 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260809 07:31:57.515450 30208 tablet_service.cc:1467] Processing CreateTablet for tablet d9cfc16f085e4df6a9b8b0c640716591 (DEFAULT_TABLE table=test-workload [id=e3f829065f584f0eaf5271c92f6e2eca]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260809 07:31:57.531268 30210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e362b890f4884247b19c99202237dfc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.531893 30199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513a5ae4a1754e57bc8d200f326a00ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.532539 30207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213881c3cf094e188ad18873996bc958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.538520 30203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793bb066bf22463e9bd510ab0984ddce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.539271 30200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98ba7543fca47bba392925a1ef848c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.542662 30208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9cfc16f085e4df6a9b8b0c640716591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.546634 30205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d773aaa943354683b0b3439ccb5bba10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.548879 30206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe2396b43db427c9f4bcaabca4d7589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.550647 30211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09825d9aebf344c1904f7bed4e5cfaaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:31:57.682155 30418 tablet_bootstrap.cc:492] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:57.686589 30417 tablet_bootstrap.cc:492] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.691999 30415 tablet_bootstrap.cc:492] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.692474 30415 ts_tablet_manager.cc:1397] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.202s user 0.015s sys 0.044s
I20260809 07:31:57.694077 30417 tablet_bootstrap.cc:654] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.695143 30415 raft_consensus.cc:348] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:57.695891 30415 raft_consensus.cc:374] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.696182 30415 raft_consensus.cc:729] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.696903 30415 consensus_queue.cc:260] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:57.699185 30149 heartbeater.cc:503] Master 127.28.87.126:39357 was elected leader, sending a full tablet report...
I20260809 07:31:57.699671 30415 ts_tablet_manager.cc:1428] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 07:31:57.699772 30418 tablet_bootstrap.cc:654] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.700649 30415 tablet_bootstrap.cc:492] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.706552 30415 tablet_bootstrap.cc:654] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.708906 30417 tablet_bootstrap.cc:492] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.709393 30417 ts_tablet_manager.cc:1397] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.006s
I20260809 07:31:57.712410 30417 raft_consensus.cc:348] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.713135 30417 raft_consensus.cc:374] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.713433 30417 raft_consensus.cc:729] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.714135 30417 consensus_queue.cc:260] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.716722 30392 heartbeater.cc:503] Master 127.28.87.126:39357 was elected leader, sending a full tablet report...
I20260809 07:31:57.717036 30417 ts_tablet_manager.cc:1428] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 07:31:57.718025 30417 tablet_bootstrap.cc:492] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.720899 30418 tablet_bootstrap.cc:492] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:57.721354 30418 ts_tablet_manager.cc:1397] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.040s user 0.012s sys 0.003s
I20260809 07:31:57.723459 30417 tablet_bootstrap.cc:654] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.723994 30418 raft_consensus.cc:348] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.724741 30418 raft_consensus.cc:374] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.725026 30418 raft_consensus.cc:729] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.725854 30418 consensus_queue.cc:260] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.750873 30415 tablet_bootstrap.cc:492] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.750823 30271 heartbeater.cc:503] Master 127.28.87.126:39357 was elected leader, sending a full tablet report...
I20260809 07:31:57.751403 30418 ts_tablet_manager.cc:1428] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.030s user 0.017s sys 0.008s
I20260809 07:31:57.751606 30415 ts_tablet_manager.cc:1397] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.051s user 0.012s sys 0.027s
I20260809 07:31:57.751734 30417 tablet_bootstrap.cc:492] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.752345 30418 tablet_bootstrap.cc:492] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:57.752362 30417 ts_tablet_manager.cc:1397] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.035s user 0.031s sys 0.003s
I20260809 07:31:57.754590 30415 raft_consensus.cc:348] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.755096 30417 raft_consensus.cc:348] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:57.755786 30415 raft_consensus.cc:374] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.755879 30417 raft_consensus.cc:374] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.756366 30415 raft_consensus.cc:729] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.756366 30417 raft_consensus.cc:729] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.757279 30415 consensus_queue.cc:260] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.758595 30418 tablet_bootstrap.cc:654] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.759415 30415 ts_tablet_manager.cc:1428] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260809 07:31:57.757218 30417 consensus_queue.cc:260] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:57.761586 30417 ts_tablet_manager.cc:1428] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260809 07:31:57.762408 30417 tablet_bootstrap.cc:492] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.764490 30415 tablet_bootstrap.cc:492] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.770507 30415 tablet_bootstrap.cc:654] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.770859 30418 tablet_bootstrap.cc:492] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:57.771311 30418 ts_tablet_manager.cc:1397] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.011s
I20260809 07:31:57.773855 30418 raft_consensus.cc:348] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.774611 30418 raft_consensus.cc:374] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.774881 30418 raft_consensus.cc:729] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.775527 30418 consensus_queue.cc:260] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.777417 30417 tablet_bootstrap.cc:654] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.777766 30418 ts_tablet_manager.cc:1428] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:31:57.778811 30418 tablet_bootstrap.cc:492] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:57.780638 30415 tablet_bootstrap.cc:492] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.781246 30415 ts_tablet_manager.cc:1397] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.006s
I20260809 07:31:57.784512 30418 tablet_bootstrap.cc:654] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.784073 30415 raft_consensus.cc:348] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.785017 30415 raft_consensus.cc:374] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.785374 30415 raft_consensus.cc:729] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.786279 30415 consensus_queue.cc:260] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.788933 30415 ts_tablet_manager.cc:1428] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:31:57.790179 30415 tablet_bootstrap.cc:492] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.796595 30415 tablet_bootstrap.cc:654] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.809616 30417 tablet_bootstrap.cc:492] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.810118 30417 ts_tablet_manager.cc:1397] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.048s user 0.007s sys 0.029s
I20260809 07:31:57.810778 30415 tablet_bootstrap.cc:492] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.811229 30415 ts_tablet_manager.cc:1397] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.007s
I20260809 07:31:57.812968 30418 tablet_bootstrap.cc:492] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:57.812700 30417 raft_consensus.cc:348] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.813330 30418 ts_tablet_manager.cc:1397] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.021s
I20260809 07:31:57.813457 30417 raft_consensus.cc:374] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.813802 30417 raft_consensus.cc:729] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.813587 30415 raft_consensus.cc:348] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.814283 30415 raft_consensus.cc:374] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.814596 30415 raft_consensus.cc:729] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.814508 30417 consensus_queue.cc:260] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.815280 30415 consensus_queue.cc:260] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.815526 30418 raft_consensus.cc:348] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.816231 30418 raft_consensus.cc:374] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.816502 30418 raft_consensus.cc:729] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.816622 30417 ts_tablet_manager.cc:1428] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260809 07:31:57.817482 30417 tablet_bootstrap.cc:492] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.817219 30418 consensus_queue.cc:260] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.819965 30415 ts_tablet_manager.cc:1428] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260809 07:31:57.820900 30415 tablet_bootstrap.cc:492] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.821780 30418 ts_tablet_manager.cc:1428] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260809 07:31:57.822628 30418 tablet_bootstrap.cc:492] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:57.823616 30417 tablet_bootstrap.cc:654] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.826537 30415 tablet_bootstrap.cc:654] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.828122 30418 tablet_bootstrap.cc:654] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.835371 30417 tablet_bootstrap.cc:492] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.835888 30417 ts_tablet_manager.cc:1397] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260809 07:31:57.839269 30417 raft_consensus.cc:348] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.840056 30417 raft_consensus.cc:374] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.840345 30417 raft_consensus.cc:729] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.842231 30417 consensus_queue.cc:260] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.843434 30415 tablet_bootstrap.cc:492] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.843886 30415 ts_tablet_manager.cc:1397] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.012s
I20260809 07:31:57.844626 30417 ts_tablet_manager.cc:1428] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 07:31:57.845525 30417 tablet_bootstrap.cc:492] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.846446 30415 raft_consensus.cc:348] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.847164 30415 raft_consensus.cc:374] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.847448 30415 raft_consensus.cc:729] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.848188 30415 consensus_queue.cc:260] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.850632 30415 ts_tablet_manager.cc:1428] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260809 07:31:57.851269 30417 tablet_bootstrap.cc:654] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.851635 30415 tablet_bootstrap.cc:492] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.852779 30423 raft_consensus.cc:482] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:57.853250 30423 raft_consensus.cc:504] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.855742 30423 leader_election.cc:288] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.856696 30423 raft_consensus.cc:482] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:57.857084 30423 raft_consensus.cc:504] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.857607 30415 tablet_bootstrap.cc:654] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.859448 30423 leader_election.cc:288] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.869688 30415 tablet_bootstrap.cc:492] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.869642 30417 tablet_bootstrap.cc:492] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.870182 30415 ts_tablet_manager.cc:1397] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.007s
I20260809 07:31:57.870183 30417 ts_tablet_manager.cc:1397] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.025s user 0.014s sys 0.003s
I20260809 07:31:57.872903 30417 raft_consensus.cc:348] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.873082 30415 raft_consensus.cc:348] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.873651 30417 raft_consensus.cc:374] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.873739 30415 raft_consensus.cc:374] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.873971 30417 raft_consensus.cc:729] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.874056 30415 raft_consensus.cc:729] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.873870 30423 raft_consensus.cc:482] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:57.874715 30423 raft_consensus.cc:504] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.874841 30417 consensus_queue.cc:260] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.874850 30415 consensus_queue.cc:260] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.876510 30423 leader_election.cc:288] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.880081 30417 ts_tablet_manager.cc:1428] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260809 07:31:57.881127 30417 tablet_bootstrap.cc:492] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.887275 30417 tablet_bootstrap.cc:654] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.892163 30415 ts_tablet_manager.cc:1428] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.022s user 0.005s sys 0.013s
I20260809 07:31:57.893064 30415 tablet_bootstrap.cc:492] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.894037 30417 tablet_bootstrap.cc:492] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.894526 30417 ts_tablet_manager.cc:1397] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260809 07:31:57.897195 30417 raft_consensus.cc:348] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.898089 30417 raft_consensus.cc:374] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.898407 30417 raft_consensus.cc:729] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.898928 30415 tablet_bootstrap.cc:654] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.899175 30417 consensus_queue.cc:260] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.907007 30417 ts_tablet_manager.cc:1428] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260809 07:31:57.908021 30417 tablet_bootstrap.cc:492] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.910198 30415 tablet_bootstrap.cc:492] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.910706 30415 ts_tablet_manager.cc:1397] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20260809 07:31:57.913797 30417 tablet_bootstrap.cc:654] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.913352 30415 raft_consensus.cc:348] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.914157 30415 raft_consensus.cc:374] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.914470 30415 raft_consensus.cc:729] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.915762 30415 consensus_queue.cc:260] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.918431 30415 ts_tablet_manager.cc:1428] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 07:31:57.919538 30415 tablet_bootstrap.cc:492] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.926159 30415 tablet_bootstrap.cc:654] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.931846 30417 tablet_bootstrap.cc:492] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.932340 30417 ts_tablet_manager.cc:1397] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.025s user 0.014s sys 0.008s
I20260809 07:31:57.933118 30415 tablet_bootstrap.cc:492] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.933545 30415 ts_tablet_manager.cc:1397] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.002s
I20260809 07:31:57.934372 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:57.934900 30417 raft_consensus.cc:348] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.935221 30350 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:57.936048 30417 raft_consensus.cc:374] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.935742 30415 raft_consensus.cc:348] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.936600 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5d2edb16624218b5f8229e63385b4d" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:57.937233 30415 raft_consensus.cc:374] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.937520 30415 raft_consensus.cc:729] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.937561 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213881c3cf094e188ad18873996bc958" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:57.938218 30349 raft_consensus.cc:2454] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:57.938200 30415 consensus_queue.cc:260] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.938854 30029 leader_election.cc:302] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:57.936928 30417 raft_consensus.cc:729] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.939307 30233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5d2edb16624218b5f8229e63385b4d" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:57.938856 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:57.943001 30452 raft_consensus.cc:2781] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:57.943137 30235 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:57.943373 30452 raft_consensus.cc:482] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:57.943691 30452 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:57.944604 30417 consensus_queue.cc:260] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.945492 30350 raft_consensus.cc:2454] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:57.950863 30452 raft_consensus.cc:504] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.952766 30417 ts_tablet_manager.cc:1428] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260809 07:31:57.940217 30415 ts_tablet_manager.cc:1428] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:31:57.940737 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213881c3cf094e188ad18873996bc958" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:57.953418 30230 raft_consensus.cc:2454] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:57.953783 30417 tablet_bootstrap.cc:492] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:57.955135 30415 tablet_bootstrap.cc:492] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:57.952838 30452 leader_election.cc:288] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.959944 30417 tablet_bootstrap.cc:654] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:57.960635 30415 tablet_bootstrap.cc:654] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
W20260809 07:31:57.964447 29992 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:31:57.969661 30425 raft_consensus.cc:482] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:57.970140 30425 raft_consensus.cc:504] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:57.972618 30425 leader_election.cc:288] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:57.943094 30029 leader_election.cc:302] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:57.973659 30452 raft_consensus.cc:2781] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:57.974031 30452 raft_consensus.cc:482] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:57.974359 30452 raft_consensus.cc:3037] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:57.974784 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:57.975366 30350 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:57.977792 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:57.978451 30230 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:57.983534 30350 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
W20260809 07:31:57.984817 30029 leader_election.cc:341] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:57.986402 30029 leader_election.cc:302] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:57.987175 30423 raft_consensus.cc:2781] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:57.987541 30423 raft_consensus.cc:482] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:57.987833 30423 raft_consensus.cc:3037] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:57.988415 30417 tablet_bootstrap.cc:492] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:57.988858 30417 ts_tablet_manager.cc:1397] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.007s
I20260809 07:31:57.991290 30452 raft_consensus.cc:504] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.993635 30452 leader_election.cc:288] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.986632 30230 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:57.985056 30415 tablet_bootstrap.cc:492] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:57.995059 30029 leader_election.cc:302] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:57.995256 30415 ts_tablet_manager.cc:1397] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.004s
I20260809 07:31:57.994247 30423 raft_consensus.cc:504] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.994244 30417 raft_consensus.cc:348] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.996613 30417 raft_consensus.cc:374] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.996862 30417 raft_consensus.cc:729] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.997293 30423 leader_election.cc:288] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:57.997606 30415 raft_consensus.cc:348] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.998296 30415 raft_consensus.cc:374] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:57.998988 30415 raft_consensus.cc:729] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:57.998802 30417 consensus_queue.cc:260] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:57.999637 30415 consensus_queue.cc:260] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.001626 30415 ts_tablet_manager.cc:1428] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260809 07:31:58.002661 30423 raft_consensus.cc:2781] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.003193 30423 raft_consensus.cc:686] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.005211 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213881c3cf094e188ad18873996bc958" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.004922 30423 consensus_queue.cc:237] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.003861 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213881c3cf094e188ad18873996bc958" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.005767 30350 raft_consensus.cc:3037] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.005970 30230 raft_consensus.cc:3037] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.006413 30418 tablet_bootstrap.cc:492] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:58.006850 30418 ts_tablet_manager.cc:1397] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.184s user 0.073s sys 0.053s
I20260809 07:31:58.007436 30425 raft_consensus.cc:482] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.007865 30425 raft_consensus.cc:504] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.009632 30425 leader_election.cc:288] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.009300 30418 raft_consensus.cc:348] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.016908 30418 raft_consensus.cc:374] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.017204 30418 raft_consensus.cc:729] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.017412 30417 ts_tablet_manager.cc:1428] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.028s user 0.004s sys 0.006s
I20260809 07:31:58.011471 30425 raft_consensus.cc:482] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.017920 30425 raft_consensus.cc:504] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.018620 30350 raft_consensus.cc:2454] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:58.019568 30415 tablet_bootstrap.cc:492] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: Bootstrap starting.
I20260809 07:31:58.019789 30425 leader_election.cc:288] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.020681 30418 consensus_queue.cc:260] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.023734 30418 ts_tablet_manager.cc:1428] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260809 07:31:58.024631 30418 tablet_bootstrap.cc:492] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:58.026458 30417 tablet_bootstrap.cc:492] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:58.026845 30230 raft_consensus.cc:2454] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:58.029417 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5d2edb16624218b5f8229e63385b4d" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.030196 30418 tablet_bootstrap.cc:654] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.032534 30415 tablet_bootstrap.cc:654] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.035308 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5d2edb16624218b5f8229e63385b4d" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.035910 30350 raft_consensus.cc:3037] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.030122 30029 leader_election.cc:302] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
W20260809 07:31:58.040161 30029 leader_election.cc:341] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.040872 30423 raft_consensus.cc:2781] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.041424 30423 raft_consensus.cc:686] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.042397 30350 raft_consensus.cc:2454] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:58.042250 30423 consensus_queue.cc:237] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.045727 30417 tablet_bootstrap.cc:654] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.048486 30029 leader_election.cc:302] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.048959 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.050307 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.050655 30452 raft_consensus.cc:2781] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.050415 30233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.054723 30425 raft_consensus.cc:482] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.058749 30415 tablet_bootstrap.cc:492] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: No bootstrap required, opened a new log
I20260809 07:31:58.059216 30415 ts_tablet_manager.cc:1397] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.003s
I20260809 07:31:58.059314 30418 tablet_bootstrap.cc:492] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
W20260809 07:31:58.059587 30289 leader_election.cc:341] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.059964 30418 ts_tablet_manager.cc:1397] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.035s user 0.015s sys 0.016s
W20260809 07:31:58.060290 30289 leader_election.cc:341] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:31:58.060892 30289 leader_election.cc:341] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.055917 30425 raft_consensus.cc:504] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.063306 30425 leader_election.cc:288] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.063236 30418 raft_consensus.cc:348] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.063041 29908 catalog_manager.cc:5495] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.064276 30418 raft_consensus.cc:374] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.064734 30418 raft_consensus.cc:729] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.065488 30418 consensus_queue.cc:260] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.066216 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.067791 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.067965 30093 raft_consensus.cc:2454] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 0.
I20260809 07:31:58.067898 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.068511 30092 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 0.
I20260809 07:31:58.069232 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.069566 30417 tablet_bootstrap.cc:492] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:58.069977 30417 ts_tablet_manager.cc:1397] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.010s
I20260809 07:31:58.069592 30293 leader_election.cc:302] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.070861 30091 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 0.
I20260809 07:31:58.071089 30293 leader_election.cc:302] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.071905 30417 raft_consensus.cc:348] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.072441 30417 raft_consensus.cc:374] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.072706 30417 raft_consensus.cc:729] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.072661 30233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.069854 30090 raft_consensus.cc:2454] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 0.
I20260809 07:31:58.073396 30233 raft_consensus.cc:2454] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 0.
I20260809 07:31:58.073735 30425 raft_consensus.cc:2781] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.073505 30417 consensus_queue.cc:260] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.074100 30425 raft_consensus.cc:482] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.074472 30289 leader_election.cc:302] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.074502 30425 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.079269 30415 raft_consensus.cc:348] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.081043 30415 raft_consensus.cc:374] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.081252 30415 raft_consensus.cc:729] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.081776 30415 consensus_queue.cc:260] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.082984 30418 ts_tablet_manager.cc:1428] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260809 07:31:58.083966 30418 tablet_bootstrap.cc:492] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:58.092310 30418 tablet_bootstrap.cc:654] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.092350 30425 raft_consensus.cc:504] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.094631 30425 leader_election.cc:288] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.098245 30452 raft_consensus.cc:686] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.101327 30423 raft_consensus.cc:482] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.101722 30452 consensus_queue.cc:237] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.101820 30423 raft_consensus.cc:504] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.102624 30474 raft_consensus.cc:2781] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.102988 30474 raft_consensus.cc:482] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.103286 30474 raft_consensus.cc:3037] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.104635 30415 ts_tablet_manager.cc:1428] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: Time spent starting tablet: real 0.038s user 0.001s sys 0.004s
I20260809 07:31:58.103767 30425 raft_consensus.cc:2781] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.106109 30293 leader_election.cc:302] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.107686 30417 ts_tablet_manager.cc:1428] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.037s user 0.008s sys 0.000s
I20260809 07:31:58.109133 30417 tablet_bootstrap.cc:492] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: Bootstrap starting.
I20260809 07:31:58.109436 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.110117 30233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.110585 30090 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.110723 30233 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.103925 30423 leader_election.cc:288] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.114750 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:58.114985 30473 raft_consensus.cc:482] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.115433 30350 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:58.115819 30478 raft_consensus.cc:2781] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.116221 30478 raft_consensus.cc:482] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.115456 30473 raft_consensus.cc:504] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.116611 30478 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.122134 30090 raft_consensus.cc:2454] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.122893 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.123502 30293 leader_election.cc:302] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.123782 30029 leader_election.cc:302] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.124714 30423 raft_consensus.cc:2781] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.125068 30423 raft_consensus.cc:482] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.125391 30423 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.123513 30230 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:58.108012 30425 raft_consensus.cc:482] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.131767 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.131728 30474 raft_consensus.cc:504] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.133533 30423 raft_consensus.cc:504] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
W20260809 07:31:58.134001 30029 leader_election.cc:341] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.134209 30479 raft_consensus.cc:2781] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.136276 30474 leader_election.cc:288] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.137219 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.137765 30230 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.138625 30417 tablet_bootstrap.cc:654] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.130255 30425 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.142649 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.143805 30235 raft_consensus.cc:3037] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.143173 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.147162 30423 leader_election.cc:288] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.151851 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:58.152456 30350 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
I20260809 07:31:58.153297 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.153627 30230 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:58.153841 30349 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.154712 30029 leader_election.cc:302] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, 46a203ae94664aadaa6699d61543aec3; no voters:
I20260809 07:31:58.155413 30423 raft_consensus.cc:2781] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.155547 30029 leader_election.cc:302] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.143050 30233 raft_consensus.cc:2454] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.146692 30473 leader_election.cc:288] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.157454 30473 raft_consensus.cc:2781] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.157758 30473 raft_consensus.cc:482] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.158135 30473 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.145612 30090 raft_consensus.cc:3037] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.169463 30349 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:58.171797 30235 raft_consensus.cc:2454] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.178480 30289 leader_election.cc:302] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.188274 30474 raft_consensus.cc:2781] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.188841 30474 raft_consensus.cc:686] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.189698 30474 consensus_queue.cc:237] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.192278 30479 raft_consensus.cc:686] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.193179 30479 consensus_queue.cc:237] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.178838 30473 raft_consensus.cc:504] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.196823 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.193853 30418 tablet_bootstrap.cc:492] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
W20260809 07:31:58.198342 30029 leader_election.cc:341] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.200678 30418 ts_tablet_manager.cc:1397] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.117s user 0.012s sys 0.004s
I20260809 07:31:58.202661 30417 tablet_bootstrap.cc:492] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: No bootstrap required, opened a new log
I20260809 07:31:58.203079 30417 ts_tablet_manager.cc:1397] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: Time spent bootstrapping tablet: real 0.094s user 0.014s sys 0.025s
I20260809 07:31:58.204064 30473 leader_election.cc:288] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.199864 30090 raft_consensus.cc:2454] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.205485 30417 raft_consensus.cc:348] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.206156 30417 raft_consensus.cc:374] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.209975 30417 raft_consensus.cc:729] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.210863 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.210798 30417 consensus_queue.cc:260] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.211474 30349 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.212208 30425 raft_consensus.cc:504] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.173280 30478 raft_consensus.cc:504] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.212845 30417 ts_tablet_manager.cc:1428] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 07:31:58.213071 30423 raft_consensus.cc:686] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.214169 30423 consensus_queue.cc:237] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.216813 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.217587 30349 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
W20260809 07:31:58.217939 30289 leader_election.cc:341] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.218544 30478 leader_election.cc:288] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.218729 30029 leader_election.cc:302] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.219250 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.219817 30090 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.220060 30494 raft_consensus.cc:2781] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.220500 30494 raft_consensus.cc:686] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.221164 30494 consensus_queue.cc:237] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.222188 29912 catalog_manager.cc:5495] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 0 to 1, leader changed from <none> to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.224622 30425 leader_election.cc:288] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.225687 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.227094 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.227717 30091 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.225479 30418 raft_consensus.cc:348] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.230567 30418 raft_consensus.cc:374] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.234705 30418 raft_consensus.cc:729] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.234709 30091 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.235431 30418 consensus_queue.cc:260] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.236310 30235 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.235680 29910 catalog_manager.cc:5495] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.236310 30293 leader_election.cc:302] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.237746 30090 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.238893 30293 leader_election.cc:302] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.237854 29910 catalog_manager.cc:5495] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.239152 29908 catalog_manager.cc:5495] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 0 to 1, leader changed from <none> to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:58.239986 30478 raft_consensus.cc:2781] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.240445 30478 raft_consensus.cc:686] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.240608 29910 catalog_manager.cc:5495] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.241227 30478 consensus_queue.cc:237] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.243010 30479 raft_consensus.cc:2781] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.243299 30423 raft_consensus.cc:482] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.243491 30479 raft_consensus.cc:686] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.242578 29910 catalog_manager.cc:5495] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.243747 30423 raft_consensus.cc:504] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.242815 30235 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 1.
I20260809 07:31:58.244342 30479 consensus_queue.cc:237] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.248358 30423 leader_election.cc:288] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.249383 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f" is_pre_election: true
I20260809 07:31:58.249502 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:58.250101 30349 raft_consensus.cc:2454] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 0.
W20260809 07:31:58.250777 30029 leader_election.cc:341] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.251492 30029 leader_election.cc:302] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.254518 30418 ts_tablet_manager.cc:1428] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.053s user 0.006s sys 0.004s
I20260809 07:31:58.255497 30418 tablet_bootstrap.cc:492] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:58.256052 30423 raft_consensus.cc:2781] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:31:58.256456 30423 raft_consensus.cc:482] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:31:58.256799 30423 raft_consensus.cc:3037] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.264812 30423 raft_consensus.cc:504] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.267052 30418 tablet_bootstrap.cc:654] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.272630 30423 leader_election.cc:288] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.273622 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.273802 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.274165 30418 tablet_bootstrap.cc:492] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:58.274268 30349 raft_consensus.cc:3037] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.274700 30418 ts_tablet_manager.cc:1397] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.002s
W20260809 07:31:58.274984 30029 leader_election.cc:341] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:31:58.279671 30418 raft_consensus.cc:348] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.279422 29912 catalog_manager.cc:5495] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 0 to 1, leader changed from <none> to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:58.280251 30349 raft_consensus.cc:2454] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 1.
I20260809 07:31:58.280421 30418 raft_consensus.cc:374] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.280771 30418 raft_consensus.cc:729] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.281102 29912 catalog_manager.cc:5495] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 0 to 1, leader changed from <none> to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:58.281468 30418 consensus_queue.cc:260] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.281401 30029 leader_election.cc:302] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.282974 30423 raft_consensus.cc:2781] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:31:58.283450 30423 raft_consensus.cc:686] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.283591 30418 ts_tablet_manager.cc:1428] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:31:58.284449 30418 tablet_bootstrap.cc:492] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:58.284229 30423 consensus_queue.cc:237] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.289742 30418 tablet_bootstrap.cc:654] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.293852 29910 catalog_manager.cc:5495] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.295677 30418 tablet_bootstrap.cc:492] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:58.296142 30418 ts_tablet_manager.cc:1397] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260809 07:31:58.297904 30418 raft_consensus.cc:348] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.298719 30418 raft_consensus.cc:374] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.298987 30418 raft_consensus.cc:729] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.299418 30418 consensus_queue.cc:260] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.301059 30418 ts_tablet_manager.cc:1428] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:31:58.301873 30418 tablet_bootstrap.cc:492] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:58.306911 30418 tablet_bootstrap.cc:654] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.310868 30418 tablet_bootstrap.cc:492] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:58.311143 30418 ts_tablet_manager.cc:1397] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 07:31:58.312762 30418 raft_consensus.cc:348] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.313180 30418 raft_consensus.cc:374] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.313371 30418 raft_consensus.cc:729] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.313853 30418 consensus_queue.cc:260] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.315233 30418 ts_tablet_manager.cc:1428] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 07:31:58.315980 30418 tablet_bootstrap.cc:492] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: Bootstrap starting.
I20260809 07:31:58.319674 30418 tablet_bootstrap.cc:654] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: Neither blocks nor log segments found. Creating new log.
I20260809 07:31:58.321095 30428 raft_consensus.cc:482] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.321525 30428 raft_consensus.cc:504] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.324385 30418 tablet_bootstrap.cc:492] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: No bootstrap required, opened a new log
I20260809 07:31:58.324802 30418 ts_tablet_manager.cc:1397] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 07:31:58.325531 30428 leader_election.cc:288] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.327164 30418 raft_consensus.cc:348] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.327872 30418 raft_consensus.cc:374] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:31:58.328117 30418 raft_consensus.cc:729] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Initialized, Role: FOLLOWER
I20260809 07:31:58.328683 30418 consensus_queue.cc:260] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.330677 30418 ts_tablet_manager.cc:1428] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260809 07:31:58.337663 30393 tablet.cc:2369] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:31:58.343555 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:58.346024 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.346874 30090 raft_consensus.cc:2379] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3987da397aeb4c19982f3328ea51314f in current term 1: Already voted for candidate a421085add2a4883856e420b4d38db00 in this term.
I20260809 07:31:58.347900 30175 leader_election.cc:302] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f; no voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00
I20260809 07:31:58.348738 30428 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.353312 30428 raft_consensus.cc:2726] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:58.356602 30349 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf"
dest_uuid: "a421085add2a4883856e420b4d38db00"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:40974
I20260809 07:31:58.357118 30349 raft_consensus.cc:595] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:31:58.366927 30349 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591"
dest_uuid: "a421085add2a4883856e420b4d38db00"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:40974
I20260809 07:31:58.367383 30349 raft_consensus.cc:595] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:31:58.382678 30090 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:37444
I20260809 07:31:58.383140 30090 raft_consensus.cc:595] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Received request to transfer leadership to TS 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.387393 30349 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10"
dest_uuid: "a421085add2a4883856e420b4d38db00"
mode: GRACEFUL
new_leader_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:40974
I20260809 07:31:58.387818 30349 raft_consensus.cc:595] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Received request to transfer leadership to TS 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.392258 30090 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:37444
I20260809 07:31:58.392831 30090 raft_consensus.cc:595] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Received request to transfer leadership to TS 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.397351 30349 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0"
dest_uuid: "a421085add2a4883856e420b4d38db00"
mode: GRACEFUL
new_leader_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:40974
I20260809 07:31:58.397784 30349 raft_consensus.cc:595] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Received request to transfer leadership to TS 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.402145 30090 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:37444
I20260809 07:31:58.402642 30090 raft_consensus.cc:595] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Received request to transfer leadership to TS a421085add2a4883856e420b4d38db00
I20260809 07:31:58.408959 30423 consensus_queue.cc:1035] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.419039 30423 consensus_queue.cc:1035] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.426275 30452 raft_consensus.cc:979] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: : Instructing follower 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.426697 30423 raft_consensus.cc:1067] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Signalling peer 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.428067 30235 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:58610
I20260809 07:31:58.428623 30235 raft_consensus.cc:482] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.428972 30235 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.433902 30235 raft_consensus.cc:504] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.435359 30235 leader_election.cc:288] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.436269 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.436395 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.436949 30090 raft_consensus.cc:3032] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.437017 30349 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.437369 30090 raft_consensus.cc:729] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.438150 30090 consensus_queue.cc:260] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.439462 30090 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.441835 30349 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.442848 30172 leader_election.cc:302] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.443637 30428 raft_consensus.cc:2781] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.444650 30090 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.444747 30428 raft_consensus.cc:686] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Leader. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:31:58.445526 30428 consensus_queue.cc:237] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.451436 29908 catalog_manager.cc:5495] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f reported cstate change: term changed from 1 to 2, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to 3987da397aeb4c19982f3328ea51314f (127.28.87.66). New cstate: current_term: 2 leader_uuid: "3987da397aeb4c19982f3328ea51314f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.484841 30423 consensus_queue.cc:1035] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.485801 30428 raft_consensus.cc:482] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.486232 30428 raft_consensus.cc:504] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.488792 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.488875 30428 leader_election.cc:288] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.489650 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:58.490466 30349 raft_consensus.cc:2379] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3987da397aeb4c19982f3328ea51314f in current term 1: Already voted for candidate 46a203ae94664aadaa6699d61543aec3 in this term.
I20260809 07:31:58.491362 30172 leader_election.cc:302] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f; no voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00
I20260809 07:31:58.492372 30428 raft_consensus.cc:3037] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.499554 30428 raft_consensus.cc:2726] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:58.500554 30423 consensus_queue.cc:1035] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.518339 30428 raft_consensus.cc:482] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:31:58.518782 30428 raft_consensus.cc:504] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.520193 30428 leader_election.cc:288] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.520974 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5d2edb16624218b5f8229e63385b4d" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a203ae94664aadaa6699d61543aec3" is_pre_election: true
I20260809 07:31:58.521111 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5d2edb16624218b5f8229e63385b4d" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a421085add2a4883856e420b4d38db00" is_pre_election: true
I20260809 07:31:58.521889 30349 raft_consensus.cc:2379] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3987da397aeb4c19982f3328ea51314f in current term 1: Already voted for candidate 46a203ae94664aadaa6699d61543aec3 in this term.
I20260809 07:31:58.522840 30172 leader_election.cc:302] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f; no voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00
I20260809 07:31:58.523438 30428 raft_consensus.cc:3037] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.527681 30428 raft_consensus.cc:2726] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:31:58.542079 30423 consensus_queue.cc:1035] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.552389 30488 consensus_queue.cc:1035] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.618181 30233 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:31:58.618407 30488 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.624869 30423 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.630055 30488 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.635663 30488 raft_consensus.cc:979] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: : Instructing follower 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.636772 30488 raft_consensus.cc:1067] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Signalling peer 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.640414 30235 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:58610
I20260809 07:31:58.640970 30235 raft_consensus.cc:482] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.641320 30235 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.642829 30423 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.646059 30478 consensus_queue.cc:1035] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.647864 30235 raft_consensus.cc:504] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.650573 30090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.651218 30090 raft_consensus.cc:3032] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.651528 30090 raft_consensus.cc:729] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.652275 30090 consensus_queue.cc:260] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.653228 30090 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.657534 30425 raft_consensus.cc:979] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: : Instructing follower 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.658104 30425 raft_consensus.cc:1067] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Signalling peer 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.660143 30230 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:58620
I20260809 07:31:58.660671 30230 raft_consensus.cc:482] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.661016 30230 raft_consensus.cc:3037] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.662951 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.663517 30349 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.664039 30479 consensus_queue.cc:1035] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.667358 30230 raft_consensus.cc:504] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.668586 30233 raft_consensus.cc:1226] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Rejecting Update request from peer a421085add2a4883856e420b4d38db00 for earlier term 1. Current term is 2. Ops: []
I20260809 07:31:58.669708 30349 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.670768 30172 leader_election.cc:302] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.671514 30428 raft_consensus.cc:2781] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.670869 30478 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.672014 30428 raft_consensus.cc:686] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Leader. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:31:58.672706 30478 consensus_queue.cc:1046] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.672797 30428 consensus_queue.cc:237] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.674151 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.674837 30230 leader_election.cc:288] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.675328 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98ba7543fca47bba392925a1ef848c0" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.674649 30425 raft_consensus.cc:3032] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.677300 30425 raft_consensus.cc:729] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.678062 30425 consensus_queue.cc:260] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.679159 30425 raft_consensus.cc:3037] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.683049 29908 catalog_manager.cc:5495] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f reported cstate change: term changed from 1 to 2, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to 3987da397aeb4c19982f3328ea51314f (127.28.87.66). New cstate: current_term: 2 leader_uuid: "3987da397aeb4c19982f3328ea51314f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.684088 30478 raft_consensus.cc:979] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:31:58.684473 30478 raft_consensus.cc:1067] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:31:58.661024 30235 leader_election.cc:288] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.662002 30090 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.688007 30093 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37418
I20260809 07:31:58.688539 30093 raft_consensus.cc:482] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.688875 30093 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.692304 30479 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.697733 30093 raft_consensus.cc:504] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.699623 30092 raft_consensus.cc:1226] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Rejecting Update request from peer a421085add2a4883856e420b4d38db00 for earlier term 1. Current term is 2. Ops: []
I20260809 07:31:58.699751 30093 leader_election.cc:288] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 2 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.701265 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.701871 30350 raft_consensus.cc:3032] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.702174 30350 raft_consensus.cc:729] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.702997 30350 consensus_queue.cc:260] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.703934 30479 consensus_queue.cc:1046] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.704725 30487 consensus_queue.cc:1035] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.708127 30350 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.711975 30349 raft_consensus.cc:2454] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.704056 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.713742 30172 leader_election.cc:302] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters: 46a203ae94664aadaa6699d61543aec3
I20260809 07:31:58.714536 30428 raft_consensus.cc:2781] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.715016 30428 raft_consensus.cc:686] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Leader. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:31:58.715778 30428 consensus_queue.cc:237] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.719097 30350 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 2.
I20260809 07:31:58.720089 30425 consensus_queue.cc:1035] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.722299 30029 leader_election.cc:302] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.723112 30423 raft_consensus.cc:2781] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.723542 30423 raft_consensus.cc:686] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.724413 30478 raft_consensus.cc:979] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:31:58.724288 30423 consensus_queue.cc:237] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.724844 30478 raft_consensus.cc:1067] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:31:58.726706 30091 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37418
I20260809 07:31:58.727216 30091 raft_consensus.cc:482] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.727572 30091 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.733557 30425 consensus_queue.cc:1035] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.739728 29908 catalog_manager.cc:5495] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f reported cstate change: term changed from 1 to 2, leader changed from a421085add2a4883856e420b4d38db00 (127.28.87.67) to 3987da397aeb4c19982f3328ea51314f (127.28.87.66). New cstate: current_term: 2 leader_uuid: "3987da397aeb4c19982f3328ea51314f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.728935 30423 consensus_queue.cc:1035] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260809 07:31:58.746670 30487 consensus_queue.cc:1035] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.744521 30091 raft_consensus.cc:504] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.752039 30093 raft_consensus.cc:1226] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Rejecting Update request from peer a421085add2a4883856e420b4d38db00 for earlier term 1. Current term is 2. Ops: []
I20260809 07:31:58.760300 30425 consensus_queue.cc:1046] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.761943 30474 raft_consensus.cc:3032] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.762485 30474 raft_consensus.cc:729] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.763675 30474 consensus_queue.cc:260] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.764987 30474 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.759644 29910 catalog_manager.cc:5495] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 1 to 2, leader changed from a421085add2a4883856e420b4d38db00 (127.28.87.67) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.777029 30349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.777384 30091 leader_election.cc:288] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 2 election: Requested vote from peers a421085add2a4883856e420b4d38db00 (127.28.87.67:39327), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.777845 30233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.798669 30349 raft_consensus.cc:2454] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 2.
I20260809 07:31:58.810577 30029 leader_election.cc:302] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters: 3987da397aeb4c19982f3328ea51314f
I20260809 07:31:58.812058 30423 raft_consensus.cc:2781] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.812784 30423 raft_consensus.cc:686] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.812973 30488 raft_consensus.cc:979] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: : Instructing follower a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:31:58.813722 30488 raft_consensus.cc:1067] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Signalling peer a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:31:58.814234 30478 raft_consensus.cc:979] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: : Instructing follower 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.814896 30478 raft_consensus.cc:1067] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Signalling peer 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:31:58.814128 30423 consensus_queue.cc:237] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:31:58.817255 30230 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:58620
I20260809 07:31:58.818020 30230 raft_consensus.cc:482] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.818570 30230 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.826599 30350 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac"
dest_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:40946
I20260809 07:31:58.827495 30350 raft_consensus.cc:482] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:31:58.827942 30350 raft_consensus.cc:3037] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.829210 30230 raft_consensus.cc:504] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.830796 30235 raft_consensus.cc:1226] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Rejecting Update request from peer a421085add2a4883856e420b4d38db00 for earlier term 1. Current term is 2. Ops: []
I20260809 07:31:58.831981 30423 consensus_queue.cc:1035] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:31:58.832998 29908 catalog_manager.cc:5495] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 1 to 2, leader changed from a421085add2a4883856e420b4d38db00 (127.28.87.67) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.833420 30474 consensus_queue.cc:1046] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.836058 30478 raft_consensus.cc:3032] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.836562 30478 raft_consensus.cc:729] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.837400 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.838198 30092 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.839184 30348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:31:58.840222 30350 raft_consensus.cc:504] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.837692 30478 consensus_queue.cc:260] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.842252 30349 raft_consensus.cc:1226] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Rejecting Update request from peer 46a203ae94664aadaa6699d61543aec3 for earlier term 1. Current term is 2. Ops: []
I20260809 07:31:58.842275 30478 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.837517 30230 leader_election.cc:288] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:31:58.846329 30423 consensus_queue.cc:1046] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:31:58.847651 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:31:58.851676 30092 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.854223 30350 leader_election.cc:288] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:31:58.855612 30175 leader_election.cc:302] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, 46a203ae94664aadaa6699d61543aec3; no voters:
I20260809 07:31:58.857213 30428 raft_consensus.cc:2781] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.857745 30428 raft_consensus.cc:686] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Leader. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:31:58.858954 30428 consensus_queue.cc:237] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.854764 30091 raft_consensus.cc:3032] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:31:58.860747 30091 raft_consensus.cc:729] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:31:58.861479 30091 consensus_queue.cc:260] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.862533 30091 raft_consensus.cc:3037] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.863667 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513a5ae4a1754e57bc8d200f326a00ac" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:31:58.864228 30230 raft_consensus.cc:3037] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:58.869071 30091 raft_consensus.cc:2454] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 2.
W20260809 07:31:58.869669 30473 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260809 07:31:58.872822 30293 leader_election.cc:302] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:31:58.873204 30230 raft_consensus.cc:2454] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 2.
I20260809 07:31:58.874068 30474 raft_consensus.cc:2781] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:31:58.874764 30474 raft_consensus.cc:686] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:31:58.875615 30474 consensus_queue.cc:237] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:31:58.881642 29908 catalog_manager.cc:5495] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f reported cstate change: term changed from 1 to 2, leader changed from a421085add2a4883856e420b4d38db00 (127.28.87.67) to 3987da397aeb4c19982f3328ea51314f (127.28.87.66). New cstate: current_term: 2 leader_uuid: "3987da397aeb4c19982f3328ea51314f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:31:58.885953 30348 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:31:58.891628 29910 catalog_manager.cc:5495] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 1 to 2, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 2 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:31:58.938767 30091 raft_consensus.cc:1261] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:58.940708 30428 consensus_queue.cc:1035] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:58.951638 30348 raft_consensus.cc:1261] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:58.953179 30428 consensus_queue.cc:1035] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:59.095655 30091 raft_consensus.cc:3037] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:59.104015 30091 raft_consensus.cc:1261] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.105654 30428 consensus_queue.cc:1035] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:59.108886 30348 raft_consensus.cc:1261] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.110698 30452 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:59.123370 30230 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:59.124734 30350 raft_consensus.cc:1261] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.127123 30544 consensus_queue.cc:1035] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:59.141067 30230 raft_consensus.cc:1261] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.143102 30452 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:59.177034 30350 raft_consensus.cc:1261] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.178534 30544 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:59.189132 30091 raft_consensus.cc:1261] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.191979 30428 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:59.227456 30350 raft_consensus.cc:1261] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.229074 30452 consensus_queue.cc:1035] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:59.239758 30230 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:31:59.249087 30230 raft_consensus.cc:1261] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.251139 30473 consensus_queue.cc:1035] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:31:59.352401 30091 raft_consensus.cc:1261] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.353796 30428 consensus_queue.cc:1035] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:59.362943 30350 raft_consensus.cc:1261] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.364828 30548 consensus_queue.cc:1035] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:59.429591 30091 raft_consensus.cc:1261] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.430837 30474 consensus_queue.cc:1035] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:31:59.438925 30230 raft_consensus.cc:1261] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:31:59.440361 30474 consensus_queue.cc:1035] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:01.430030 29021 auto_leader_rebalancer-test.cc:339] The leader distribution is
I20260809 07:32:01.430663 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:01.446645 30091 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:37448
I20260809 07:32:01.447100 30091 raft_consensus.cc:595] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Received request to transfer leadership to TS a421085add2a4883856e420b4d38db00
I20260809 07:32:01.449008 30091 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e362b890f4884247b19c99202237dfc2"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:37448
I20260809 07:32:01.449410 30091 raft_consensus.cc:595] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Received request to transfer leadership to TS a421085add2a4883856e420b4d38db00
I20260809 07:32:01.450167 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 07:32:01.450606 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:01.697510 30488 raft_consensus.cc:979] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: : Instructing follower a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:01.697965 30551 raft_consensus.cc:1067] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Signalling peer a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:01.699522 30348 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591"
dest_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:40946
I20260809 07:32:01.700011 30348 raft_consensus.cc:482] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:01.700295 30348 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:01.704813 30348 raft_consensus.cc:504] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:01.706337 30348 leader_election.cc:288] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 3 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:32:01.707228 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:32:01.707372 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:01.707717 30091 raft_consensus.cc:3032] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:01.707860 30230 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:01.707988 30091 raft_consensus.cc:729] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:01.708657 30091 consensus_queue.cc:260] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:01.709532 30091 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:01.712121 30230 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 3.
I20260809 07:32:01.713016 30289 leader_election.cc:302] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:01.713653 30552 raft_consensus.cc:2781] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:01.714402 30091 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 3.
I20260809 07:32:01.714641 30552 raft_consensus.cc:686] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:32:01.715447 30552 consensus_queue.cc:237] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:01.721655 29910 catalog_manager.cc:5495] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 2 to 3, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:01.777525 30488 raft_consensus.cc:979] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: : Instructing follower a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:01.777930 30551 raft_consensus.cc:1067] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Signalling peer a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:01.779255 30348 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e362b890f4884247b19c99202237dfc2"
dest_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:40946
I20260809 07:32:01.779718 30348 raft_consensus.cc:482] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:01.779981 30348 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:01.784612 30348 raft_consensus.cc:504] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:01.786116 30348 leader_election.cc:288] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:32:01.786935 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:32:01.787168 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:01.787433 30091 raft_consensus.cc:3032] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:01.787644 30230 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:01.787724 30091 raft_consensus.cc:729] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:01.788407 30091 consensus_queue.cc:260] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:01.789294 30091 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:01.792224 30230 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 2.
I20260809 07:32:01.793221 30289 leader_election.cc:302] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:01.793906 30552 raft_consensus.cc:2781] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:01.794281 30552 raft_consensus.cc:686] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:32:01.794531 30091 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 2.
I20260809 07:32:01.795056 30552 consensus_queue.cc:237] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:01.801614 29910 catalog_manager.cc:5495] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 1 to 2, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 2 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:02.213755 30230 raft_consensus.cc:1261] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:02.214946 30552 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:02.223857 30091 raft_consensus.cc:1261] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:02.225229 30552 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:02.283672 30230 raft_consensus.cc:1261] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:02.284687 30552 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:02.293224 30091 raft_consensus.cc:1261] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:02.295125 30560 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:02.451439 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:02.457727 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:02.458130 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:03.458961 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:03.464692 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:03.465081 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:04.465790 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:04.472266 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:04.472726 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:05.473546 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:05.479650 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:05.480060 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:06.481118 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:06.486934 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:06.487342 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:07.488191 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:07.493499 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:07.493921 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:08.494684 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:08.499636 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:08.500036 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:09.500859 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:09.505997 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:09.506477 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:10.507257 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:10.512470 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:10.512884 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:11.541167 29021 auto_leader_rebalancer-test.cc:358] The leader distribution is
I20260809 07:32:11.547442 30350 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e362b890f4884247b19c99202237dfc2"
dest_uuid: "a421085add2a4883856e420b4d38db00"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:40974
I20260809 07:32:11.548103 30350 raft_consensus.cc:595] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:32:11.554612 30350 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591"
dest_uuid: "a421085add2a4883856e420b4d38db00"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:40974
I20260809 07:32:11.555122 30350 raft_consensus.cc:595] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:32:11.576467 30235 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:49500
I20260809 07:32:11.576941 30235 raft_consensus.cc:595] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:32:11.581166 30235 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:49500
I20260809 07:32:11.581616 30235 raft_consensus.cc:595] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:32:11.585482 30235 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
mode: GRACEFUL
new_leader_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:49500
I20260809 07:32:11.585920 30235 raft_consensus.cc:595] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Received request to transfer leadership to TS 46a203ae94664aadaa6699d61543aec3
I20260809 07:32:11.646057 30602 raft_consensus.cc:979] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.646602 30606 raft_consensus.cc:1067] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.648016 30092 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e362b890f4884247b19c99202237dfc2"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37418
I20260809 07:32:11.648494 30092 raft_consensus.cc:482] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:11.648938 30092 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.654908 30092 raft_consensus.cc:504] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.656657 30092 leader_election.cc:288] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:11.657401 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:11.657670 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:11.658152 30235 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.658281 30350 raft_consensus.cc:3032] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:11.658679 30350 raft_consensus.cc:729] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:32:11.659432 30350 consensus_queue.cc:260] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.660696 30350 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.663522 30235 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.664705 30029 leader_election.cc:302] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, 46a203ae94664aadaa6699d61543aec3; no voters:
I20260809 07:32:11.665582 30598 raft_consensus.cc:2781] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:11.666079 30350 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.666559 30598 raft_consensus.cc:686] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:11.667433 30598 consensus_queue.cc:237] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.675431 29910 catalog_manager.cc:5495] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 2 to 3, leader changed from a421085add2a4883856e420b4d38db00 (127.28.87.67) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:11.788726 30596 raft_consensus.cc:979] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.789211 30615 raft_consensus.cc:1067] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.790577 30092 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37432
I20260809 07:32:11.791036 30092 raft_consensus.cc:482] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:11.791366 30092 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.796794 30092 raft_consensus.cc:504] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.798266 30092 leader_election.cc:288] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:11.798992 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:11.799180 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d773aaa943354683b0b3439ccb5bba10" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:11.799597 30235 raft_consensus.cc:3032] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:11.799678 30350 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.799994 30235 raft_consensus.cc:729] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:32:11.800781 30235 consensus_queue.cc:260] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.802237 30596 raft_consensus.cc:979] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.802196 30235 raft_consensus.cc:3037] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.802747 30596 raft_consensus.cc:1067] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.803995 30092 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37432
I20260809 07:32:11.804421 30092 raft_consensus.cc:482] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:11.804683 30092 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.806047 30350 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.807178 30029 leader_election.cc:302] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:11.807924 30598 raft_consensus.cc:2781] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:11.808610 30235 raft_consensus.cc:2454] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.809670 30598 raft_consensus.cc:686] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:11.810619 30598 consensus_queue.cc:237] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.811715 30092 raft_consensus.cc:504] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.813643 30092 leader_election.cc:288] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:11.814606 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:11.814235 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:11.815191 30350 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.815277 30235 raft_consensus.cc:3032] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:11.815649 30235 raft_consensus.cc:729] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:32:11.816334 30235 consensus_queue.cc:260] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.817459 30235 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.818840 29910 catalog_manager.cc:5495] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 2 to 3, leader changed from 3987da397aeb4c19982f3328ea51314f (127.28.87.66) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:11.821259 30350 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.822242 30029 leader_election.cc:302] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:11.822991 30598 raft_consensus.cc:2781] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:11.823282 30235 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.823519 30598 raft_consensus.cc:686] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:11.825203 30598 consensus_queue.cc:237] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.830921 30596 raft_consensus.cc:979] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.831298 30615 raft_consensus.cc:1067] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.831936 29910 catalog_manager.cc:5495] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 2 to 3, leader changed from 3987da397aeb4c19982f3328ea51314f (127.28.87.66) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:11.832731 30092 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37432
I20260809 07:32:11.833235 30092 raft_consensus.cc:482] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:11.833597 30092 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.838459 30092 raft_consensus.cc:504] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.840185 30092 leader_election.cc:288] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:11.840802 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:11.840982 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793bb066bf22463e9bd510ab0984ddce" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:11.841332 30235 raft_consensus.cc:3032] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:11.841459 30350 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.841581 30235 raft_consensus.cc:729] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:32:11.842264 30235 consensus_queue.cc:260] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.843173 30235 raft_consensus.cc:3037] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:11.845695 30350 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.846556 30029 leader_election.cc:302] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a203ae94664aadaa6699d61543aec3, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:11.847131 30598 raft_consensus.cc:2781] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:11.847573 30598 raft_consensus.cc:686] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:11.847771 30235 raft_consensus.cc:2454] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 3.
I20260809 07:32:11.848480 30598 consensus_queue.cc:237] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.854682 29910 catalog_manager.cc:5495] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 2 to 3, leader changed from 3987da397aeb4c19982f3328ea51314f (127.28.87.66) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:11.907284 30602 raft_consensus.cc:979] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: : Instructing follower 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.907750 30606 raft_consensus.cc:1067] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Signalling peer 46a203ae94664aadaa6699d61543aec3 to start an election
I20260809 07:32:11.909171 30092 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
from {username='slave'} at 127.0.0.1:37418
I20260809 07:32:11.909722 30092 raft_consensus.cc:482] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:11.910069 30092 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:11.914472 30092 raft_consensus.cc:504] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.915858 30092 leader_election.cc:288] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 4 election: Requested vote from peers 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:11.916785 30350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:11.916690 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9cfc16f085e4df6a9b8b0c640716591" candidate_uuid: "46a203ae94664aadaa6699d61543aec3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:11.917312 30350 raft_consensus.cc:3032] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:11.917376 30230 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:11.917582 30350 raft_consensus.cc:729] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:32:11.918188 30350 consensus_queue.cc:260] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.919113 30350 raft_consensus.cc:3037] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:11.922079 30230 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 4.
I20260809 07:32:11.922958 30029 leader_election.cc:302] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, 46a203ae94664aadaa6699d61543aec3; no voters:
I20260809 07:32:11.923034 30350 raft_consensus.cc:2454] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a203ae94664aadaa6699d61543aec3 in term 4.
I20260809 07:32:11.923719 30598 raft_consensus.cc:2781] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:11.924165 30598 raft_consensus.cc:686] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 4 LEADER]: Becoming Leader. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:11.924897 30598 consensus_queue.cc:237] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:11.931448 29910 catalog_manager.cc:5495] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 reported cstate change: term changed from 3 to 4, leader changed from a421085add2a4883856e420b4d38db00 (127.28.87.67) to 46a203ae94664aadaa6699d61543aec3 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "46a203ae94664aadaa6699d61543aec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:12.208310 30350 raft_consensus.cc:1261] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.209502 30598 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.221529 30230 raft_consensus.cc:1261] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.224984 30598 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:32:12.229058 30350 raft_consensus.cc:1261] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.230703 30598 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.243435 30230 raft_consensus.cc:1261] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.244683 30598 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.296173 30350 raft_consensus.cc:1261] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.297289 30598 consensus_queue.cc:1035] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.306288 30230 raft_consensus.cc:1261] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.307951 30598 consensus_queue.cc:1035] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.337839 30350 raft_consensus.cc:1261] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.338996 30598 consensus_queue.cc:1035] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.347258 30230 raft_consensus.cc:1261] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:12.348578 30598 consensus_queue.cc:1035] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:12.365111 30350 raft_consensus.cc:1261] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:32:12.366151 30598 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:32:12.373678 30230 raft_consensus.cc:1261] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Refusing update from remote peer 46a203ae94664aadaa6699d61543aec3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:32:12.375030 30609 consensus_queue.cc:1035] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:32:14.671080 29021 auto_leader_rebalancer-test.cc:373] The leader distribution is
I20260809 07:32:14.671687 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:14.677922 30092 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:37448
I20260809 07:32:14.678424 30092 raft_consensus.cc:595] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Received request to transfer leadership to TS a421085add2a4883856e420b4d38db00
I20260809 07:32:14.680379 30092 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "213881c3cf094e188ad18873996bc958"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:37448
I20260809 07:32:14.680775 30092 raft_consensus.cc:595] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Received request to transfer leadership to TS 3987da397aeb4c19982f3328ea51314f
I20260809 07:32:14.682585 30092 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:37448
I20260809 07:32:14.682982 30092 raft_consensus.cc:595] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Received request to transfer leadership to TS 3987da397aeb4c19982f3328ea51314f
I20260809 07:32:14.684656 30092 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e362b890f4884247b19c99202237dfc2"
dest_uuid: "46a203ae94664aadaa6699d61543aec3"
mode: GRACEFUL
new_leader_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:37448
I20260809 07:32:14.685060 30092 raft_consensus.cc:595] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Received request to transfer leadership to TS a421085add2a4883856e420b4d38db00
I20260809 07:32:14.685827 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 07:32:14.686371 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:14.687996 30647 raft_consensus.cc:979] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: : Instructing follower a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:14.688397 30648 raft_consensus.cc:1067] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Signalling peer a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:14.689692 30348 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf"
dest_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:40946
I20260809 07:32:14.690053 30348 raft_consensus.cc:482] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:14.690300 30348 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:14.694537 30348 raft_consensus.cc:504] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:32:14.695996 30348 leader_election.cc:288] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 3 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:32:14.696779 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:32:14.696938 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09825d9aebf344c1904f7bed4e5cfaaf" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:14.697292 30092 raft_consensus.cc:3032] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:14.697368 30230 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:14.697559 30092 raft_consensus.cc:729] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:14.698170 30092 consensus_queue.cc:260] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:32:14.699043 30092 raft_consensus.cc:3037] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:14.701411 30230 raft_consensus.cc:2454] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 3.
I20260809 07:32:14.702380 30289 leader_election.cc:302] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:14.703030 30606 raft_consensus.cc:2781] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:14.703123 30092 raft_consensus.cc:2454] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 3.
I20260809 07:32:14.704136 30606 raft_consensus.cc:686] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:32:14.704874 30606 consensus_queue.cc:237] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } }
I20260809 07:32:14.713362 29908 catalog_manager.cc:5495] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 2 to 3, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:14.831377 30648 raft_consensus.cc:979] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: : Instructing follower 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:32:14.831845 30650 raft_consensus.cc:1067] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Signalling peer 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:32:14.833019 30230 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "213881c3cf094e188ad18873996bc958"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:58610
I20260809 07:32:14.833446 30230 raft_consensus.cc:482] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:14.833694 30230 raft_consensus.cc:3037] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:14.837801 30230 raft_consensus.cc:504] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:14.839496 30230 leader_election.cc:288] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:14.840225 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213881c3cf094e188ad18873996bc958" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:32:14.840812 30092 raft_consensus.cc:3032] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:14.840521 30348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213881c3cf094e188ad18873996bc958" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:14.841418 30092 raft_consensus.cc:729] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:14.841557 30348 raft_consensus.cc:3037] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:14.842187 30092 consensus_queue.cc:260] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:14.843412 30092 raft_consensus.cc:3037] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:14.847851 30348 raft_consensus.cc:2454] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:32:14.849006 30172 leader_election.cc:302] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:14.849397 30092 raft_consensus.cc:2454] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 2.
I20260809 07:32:14.849861 30615 raft_consensus.cc:2781] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:14.851052 30615 raft_consensus.cc:686] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Becoming Leader. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:32:14.851727 30615 consensus_queue.cc:237] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:14.858173 29908 catalog_manager.cc:5495] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f reported cstate change: term changed from 1 to 2, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to 3987da397aeb4c19982f3328ea51314f (127.28.87.66). New cstate: current_term: 2 leader_uuid: "3987da397aeb4c19982f3328ea51314f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:15.041222 30647 raft_consensus.cc:979] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: : Instructing follower 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:32:15.041677 30650 raft_consensus.cc:1067] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Signalling peer 3987da397aeb4c19982f3328ea51314f to start an election
I20260809 07:32:15.042991 30230 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589"
dest_uuid: "3987da397aeb4c19982f3328ea51314f"
from {username='slave'} at 127.0.0.1:58610
I20260809 07:32:15.043535 30230 raft_consensus.cc:482] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:15.043787 30230 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:15.047931 30230 raft_consensus.cc:504] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:15.049116 30230 leader_election.cc:288] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 4 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), a421085add2a4883856e420b4d38db00 (127.28.87.67:39327)
I20260809 07:32:15.049878 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:32:15.050099 30348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe2396b43db427c9f4bcaabca4d7589" candidate_uuid: "3987da397aeb4c19982f3328ea51314f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a421085add2a4883856e420b4d38db00"
I20260809 07:32:15.050416 30092 raft_consensus.cc:3032] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:15.050668 30348 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:15.050693 30092 raft_consensus.cc:729] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:15.051348 30092 consensus_queue.cc:260] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:15.052152 30092 raft_consensus.cc:3037] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:15.054847 30348 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 4.
I20260809 07:32:15.055758 30172 leader_election.cc:302] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:15.056218 30092 raft_consensus.cc:2454] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3987da397aeb4c19982f3328ea51314f in term 4.
I20260809 07:32:15.056445 30615 raft_consensus.cc:2781] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:15.056947 30615 raft_consensus.cc:686] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 4 LEADER]: Becoming Leader. State: Replica: 3987da397aeb4c19982f3328ea51314f, State: Running, Role: LEADER
I20260809 07:32:15.057765 30615 consensus_queue.cc:237] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:15.064860 29908 catalog_manager.cc:5495] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f reported cstate change: term changed from 3 to 4, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to 3987da397aeb4c19982f3328ea51314f (127.28.87.66). New cstate: current_term: 4 leader_uuid: "3987da397aeb4c19982f3328ea51314f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:15.186685 30647 raft_consensus.cc:979] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: : Instructing follower a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:15.187112 30650 raft_consensus.cc:1067] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Signalling peer a421085add2a4883856e420b4d38db00 to start an election
I20260809 07:32:15.188370 30348 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e362b890f4884247b19c99202237dfc2"
dest_uuid: "a421085add2a4883856e420b4d38db00"
from {username='slave'} at 127.0.0.1:40946
I20260809 07:32:15.188753 30348 raft_consensus.cc:482] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:32:15.188966 30348 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:15.192898 30348 raft_consensus.cc:504] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:15.194085 30348 leader_election.cc:288] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 4 election: Requested vote from peers 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055), 3987da397aeb4c19982f3328ea51314f (127.28.87.66:36843)
I20260809 07:32:15.194885 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "46a203ae94664aadaa6699d61543aec3"
I20260809 07:32:15.195104 30230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e362b890f4884247b19c99202237dfc2" candidate_uuid: "a421085add2a4883856e420b4d38db00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3987da397aeb4c19982f3328ea51314f"
I20260809 07:32:15.195415 30092 raft_consensus.cc:3032] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:15.195580 30230 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:15.195708 30092 raft_consensus.cc:729] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 46a203ae94664aadaa6699d61543aec3, State: Running, Role: LEADER
I20260809 07:32:15.196381 30092 consensus_queue.cc:260] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:15.197464 30092 raft_consensus.cc:3037] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:15.201388 30230 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 4.
I20260809 07:32:15.202359 30289 leader_election.cc:302] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3987da397aeb4c19982f3328ea51314f, a421085add2a4883856e420b4d38db00; no voters:
I20260809 07:32:15.203104 30606 raft_consensus.cc:2781] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:15.203377 30092 raft_consensus.cc:2454] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a421085add2a4883856e420b4d38db00 in term 4.
I20260809 07:32:15.203526 30606 raft_consensus.cc:686] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 4 LEADER]: Becoming Leader. State: Replica: a421085add2a4883856e420b4d38db00, State: Running, Role: LEADER
I20260809 07:32:15.204352 30606 consensus_queue.cc:237] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } }
I20260809 07:32:15.206741 30230 raft_consensus.cc:1261] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:15.207897 30652 consensus_queue.cc:1035] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:32:15.211889 29908 catalog_manager.cc:5495] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 reported cstate change: term changed from 3 to 4, leader changed from 46a203ae94664aadaa6699d61543aec3 (127.28.87.65) to a421085add2a4883856e420b4d38db00 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "a421085add2a4883856e420b4d38db00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:15.224932 30092 raft_consensus.cc:1261] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:15.226063 30606 consensus_queue.cc:1035] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:15.294597 30348 raft_consensus.cc:1261] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:15.295746 30615 consensus_queue.cc:1035] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:15.304471 30092 raft_consensus.cc:1261] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:15.306061 30615 consensus_queue.cc:1035] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:15.467242 30092 raft_consensus.cc:1261] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:32:15.468304 30615 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:32:15.475957 30348 raft_consensus.cc:1261] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Refusing update from remote peer 3987da397aeb4c19982f3328ea51314f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:32:15.477535 30675 consensus_queue.cc:1035] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a421085add2a4883856e420b4d38db00" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260809 07:32:15.572625 29966 debug-util.cc:398] Leaking SignalData structure 0x7b080004f7a0 after lost signal to thread 29026
W20260809 07:32:15.573414 29966 debug-util.cc:398] Leaking SignalData structure 0x7b080004fe80 after lost signal to thread 29971
W20260809 07:32:15.574174 29966 debug-util.cc:398] Leaking SignalData structure 0x7b080014ac00 after lost signal to thread 30148
W20260809 07:32:15.574880 29966 debug-util.cc:398] Leaking SignalData structure 0x7b080019f900 after lost signal to thread 30270
W20260809 07:32:15.575562 29966 debug-util.cc:398] Leaking SignalData structure 0x7b08001eefe0 after lost signal to thread 30391
I20260809 07:32:15.691319 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:15.702296 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:15.702793 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:15.747561 30092 raft_consensus.cc:1261] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:32:15.749114 30691 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a203ae94664aadaa6699d61543aec3" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:32:15.769512 30230 raft_consensus.cc:1261] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Refusing update from remote peer a421085add2a4883856e420b4d38db00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:32:15.771160 30691 consensus_queue.cc:1035] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3987da397aeb4c19982f3328ea51314f" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:32:16.703624 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:16.709900 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:16.710492 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:17.711359 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:17.716442 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:17.716787 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:18.717638 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:18.722949 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:18.723294 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:19.724151 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:19.729324 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:19.729664 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:20.730546 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:20.736733 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:20.737082 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:21.737843 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:21.743005 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:21.743351 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:22.744156 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:22.749385 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:22.749809 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:23.750684 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:32:23.756003 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 07:32:23.756414 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:32:24.783646 29021 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20260809 07:32:24.784138 29021 tablet_server.cc:178] TabletServer@127.28.87.65:0 shutting down...
I20260809 07:32:24.804216 29021 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 07:32:24.804906 29021 tablet_replica.cc:330] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.805742 29021 raft_consensus.cc:2229] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.806416 29021 raft_consensus.cc:2258] T abe2396b43db427c9f4bcaabca4d7589 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.808192 29021 tablet_replica.cc:330] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.808646 29021 raft_consensus.cc:2229] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 4 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.809350 29021 raft_consensus.cc:2258] T d9cfc16f085e4df6a9b8b0c640716591 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.810952 29021 tablet_replica.cc:330] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.811444 29021 raft_consensus.cc:2229] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.811812 29021 raft_consensus.cc:2258] T f98ba7543fca47bba392925a1ef848c0 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.813390 29021 tablet_replica.cc:330] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.813822 29021 raft_consensus.cc:2229] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.814196 29021 raft_consensus.cc:2258] T 213881c3cf094e188ad18873996bc958 P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.815728 29021 tablet_replica.cc:330] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.816191 29021 raft_consensus.cc:2229] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.816901 29021 raft_consensus.cc:2258] T d773aaa943354683b0b3439ccb5bba10 P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.818444 29021 tablet_replica.cc:330] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.818918 29021 raft_consensus.cc:2229] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.819844 29021 raft_consensus.cc:2258] T 793bb066bf22463e9bd510ab0984ddce P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.821779 29021 tablet_replica.cc:330] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.822387 29021 raft_consensus.cc:2229] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.822901 29021 raft_consensus.cc:2258] T 513a5ae4a1754e57bc8d200f326a00ac P 46a203ae94664aadaa6699d61543aec3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.824921 29021 tablet_replica.cc:330] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.825488 29021 raft_consensus.cc:2229] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.825930 29021 raft_consensus.cc:2258] T e362b890f4884247b19c99202237dfc2 P 46a203ae94664aadaa6699d61543aec3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.827857 29021 tablet_replica.cc:330] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.828446 29021 raft_consensus.cc:2229] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.829264 29021 raft_consensus.cc:2258] T 8f5d2edb16624218b5f8229e63385b4d P 46a203ae94664aadaa6699d61543aec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.830861 29021 tablet_replica.cc:330] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3: stopping tablet replica
I20260809 07:32:24.831251 29021 raft_consensus.cc:2229] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.831632 29021 raft_consensus.cc:2258] T 09825d9aebf344c1904f7bed4e5cfaaf P 46a203ae94664aadaa6699d61543aec3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260809 07:32:24.839205 30175 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260809 07:32:24.842156 30175 consensus_peers.cc:487] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f -> Peer 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055): Couldn't send request to peer 46a203ae94664aadaa6699d61543aec3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:24.861132 29021 tablet_server.cc:194] TabletServer@127.28.87.65:0 shutdown complete.
W20260809 07:32:24.867360 30293 consensus_peers.cc:487] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 -> Peer 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055): Couldn't send request to peer 46a203ae94664aadaa6699d61543aec3. Status: Network error: Client connection negotiation failed: client connection to 127.28.87.65:37055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:24.879796 29021 tablet_server.cc:178] TabletServer@127.28.87.66:0 shutting down...
W20260809 07:32:24.892452 30293 consensus_peers.cc:487] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 -> Peer 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055): Couldn't send request to peer 46a203ae94664aadaa6699d61543aec3. Status: Network error: Client connection negotiation failed: client connection to 127.28.87.65:37055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:24.905555 29021 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 07:32:24.906258 29021 tablet_replica.cc:330] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.907078 29021 raft_consensus.cc:2229] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 4 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.908066 29021 raft_consensus.cc:2258] T abe2396b43db427c9f4bcaabca4d7589 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Raft consensus is shut down!
W20260809 07:32:24.909085 30175 consensus_peers.cc:487] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f -> Peer 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055): Couldn't send request to peer 46a203ae94664aadaa6699d61543aec3. Status: Network error: Client connection negotiation failed: client connection to 127.28.87.65:37055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:24.910270 29021 tablet_replica.cc:330] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.910816 29021 raft_consensus.cc:2229] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.911181 29021 raft_consensus.cc:2258] T 8f5d2edb16624218b5f8229e63385b4d P 3987da397aeb4c19982f3328ea51314f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.912856 29021 tablet_replica.cc:330] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.913313 29021 raft_consensus.cc:2229] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.913635 29021 raft_consensus.cc:2258] T d9cfc16f085e4df6a9b8b0c640716591 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.915510 29021 tablet_replica.cc:330] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.916105 29021 raft_consensus.cc:2229] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.916646 29021 raft_consensus.cc:2258] T 09825d9aebf344c1904f7bed4e5cfaaf P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.918622 29021 tablet_replica.cc:330] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.919082 29021 raft_consensus.cc:2229] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.919457 29021 raft_consensus.cc:2258] T 513a5ae4a1754e57bc8d200f326a00ac P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.921141 29021 tablet_replica.cc:330] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.921650 29021 raft_consensus.cc:2229] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.921988 29021 raft_consensus.cc:2258] T e362b890f4884247b19c99202237dfc2 P 3987da397aeb4c19982f3328ea51314f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.923790 29021 tablet_replica.cc:330] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.924252 29021 raft_consensus.cc:2229] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.924978 29021 raft_consensus.cc:2258] T f98ba7543fca47bba392925a1ef848c0 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.926580 29021 tablet_replica.cc:330] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.927007 29021 raft_consensus.cc:2229] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 2 LEADER]: Raft consensus shutting down.
I20260809 07:32:24.927688 29021 raft_consensus.cc:2258] T 213881c3cf094e188ad18873996bc958 P 3987da397aeb4c19982f3328ea51314f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.929157 29021 tablet_replica.cc:330] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.929584 29021 raft_consensus.cc:2229] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.929927 29021 raft_consensus.cc:2258] T d773aaa943354683b0b3439ccb5bba10 P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:24.931406 29021 tablet_replica.cc:330] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f: stopping tablet replica
I20260809 07:32:24.931842 29021 raft_consensus.cc:2229] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:24.932233 29021 raft_consensus.cc:2258] T 793bb066bf22463e9bd510ab0984ddce P 3987da397aeb4c19982f3328ea51314f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260809 07:32:24.958799 30293 consensus_peers.cc:487] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 -> Peer 46a203ae94664aadaa6699d61543aec3 (127.28.87.65:37055): Couldn't send request to peer 46a203ae94664aadaa6699d61543aec3. Status: Network error: Client connection negotiation failed: client connection to 127.28.87.65:37055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:24.960026 29021 tablet_server.cc:194] TabletServer@127.28.87.66:0 shutdown complete.
I20260809 07:32:24.977950 29021 tablet_server.cc:178] TabletServer@127.28.87.67:0 shutting down...
I20260809 07:32:25.000347 29021 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 07:32:25.000945 29021 tablet_replica.cc:330] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.001543 29021 raft_consensus.cc:2229] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.002056 29021 raft_consensus.cc:2258] T abe2396b43db427c9f4bcaabca4d7589 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.003865 29021 tablet_replica.cc:330] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.004364 29021 raft_consensus.cc:2229] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.004732 29021 raft_consensus.cc:2258] T 8f5d2edb16624218b5f8229e63385b4d P a421085add2a4883856e420b4d38db00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.006390 29021 tablet_replica.cc:330] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.006850 29021 raft_consensus.cc:2229] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.007218 29021 raft_consensus.cc:2258] T d9cfc16f085e4df6a9b8b0c640716591 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.008767 29021 tablet_replica.cc:330] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.009255 29021 raft_consensus.cc:2229] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.009667 29021 raft_consensus.cc:2258] T f98ba7543fca47bba392925a1ef848c0 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.011204 29021 tablet_replica.cc:330] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.011641 29021 raft_consensus.cc:2229] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.011996 29021 raft_consensus.cc:2258] T 213881c3cf094e188ad18873996bc958 P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.013473 29021 tablet_replica.cc:330] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.013936 29021 raft_consensus.cc:2229] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.014408 29021 raft_consensus.cc:2258] T d773aaa943354683b0b3439ccb5bba10 P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.015931 29021 tablet_replica.cc:330] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.016366 29021 raft_consensus.cc:2229] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 07:32:25.016765 29021 raft_consensus.cc:2258] T 793bb066bf22463e9bd510ab0984ddce P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.018236 29021 tablet_replica.cc:330] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.018705 29021 raft_consensus.cc:2229] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 3 LEADER]: Raft consensus shutting down.
I20260809 07:32:25.019470 29021 raft_consensus.cc:2258] T 09825d9aebf344c1904f7bed4e5cfaaf P a421085add2a4883856e420b4d38db00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.021164 29021 tablet_replica.cc:330] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.021649 29021 raft_consensus.cc:2229] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 2 LEADER]: Raft consensus shutting down.
I20260809 07:32:25.022444 29021 raft_consensus.cc:2258] T 513a5ae4a1754e57bc8d200f326a00ac P a421085add2a4883856e420b4d38db00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.024163 29021 tablet_replica.cc:330] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00: stopping tablet replica
I20260809 07:32:25.024747 29021 raft_consensus.cc:2229] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 4 LEADER]: Raft consensus shutting down.
I20260809 07:32:25.025446 29021 raft_consensus.cc:2258] T e362b890f4884247b19c99202237dfc2 P a421085add2a4883856e420b4d38db00 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.053301 29021 tablet_server.cc:194] TabletServer@127.28.87.67:0 shutdown complete.
I20260809 07:32:25.072176 29021 master.cc:537] Master@127.28.87.126:39357 shutting down...
I20260809 07:32:25.092651 29021 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:32:25.093216 29021 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:32:25.093541 29021 tablet_replica.cc:330] T 00000000000000000000000000000000 P ad53a97a568041579292455f0df871f3: stopping tablet replica
I20260809 07:32:25.126055 29021 master.cc:552] Master@127.28.87.126:39357 shutdown complete.
3987da397aeb4c19982f3328ea51314f 4
46a203ae94664aadaa6699d61543aec3 5
a421085add2a4883856e420b4d38db00 1
3987da397aeb4c19982f3328ea51314f 4
46a203ae94664aadaa6699d61543aec3 3
a421085add2a4883856e420b4d38db00 3
3987da397aeb4c19982f3328ea51314f 1
46a203ae94664aadaa6699d61543aec3 8
a421085add2a4883856e420b4d38db00 1
3987da397aeb4c19982f3328ea51314f 3
46a203ae94664aadaa6699d61543aec3 4
a421085add2a4883856e420b4d38db00 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28333 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260809 07:32:25.164860 29021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.28.87.126:41365
I20260809 07:32:25.165933 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:32:25.171849 30711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.172030 30712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.172587 30714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:32:25.173164 29021 server_base.cc:1034] running on GCE node
I20260809 07:32:25.174100 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:32:25.174278 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:32:25.174437 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260745174425 us; error 0 us; skew 500 ppm
I20260809 07:32:25.174897 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:32:25.180186 29021 webserver.cc:460] Webserver started at http://127.28.87.126:46301/ using document root <none> and password file <none>
I20260809 07:32:25.180598 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:32:25.180747 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:32:25.180982 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:32:25.182092 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/master-0-root/instance:
uuid: "8bb995981efc41a7a744904b9759529a"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.186295 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260809 07:32:25.189436 30719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.190104 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 07:32:25.190395 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/master-0-root
uuid: "8bb995981efc41a7a744904b9759529a"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.190653 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:32:25.203040 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:32:25.204218 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:32:25.241099 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.126:41365
I20260809 07:32:25.241204 30803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.126:41365 every 8 connection(s)
I20260809 07:32:25.244778 30806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.254251 30806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Bootstrap starting.
I20260809 07:32:25.258572 30806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:25.262660 30806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: No bootstrap required, opened a new log
I20260809 07:32:25.264573 30806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb995981efc41a7a744904b9759529a" member_type: VOTER }
I20260809 07:32:25.264971 30806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:25.265144 30806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bb995981efc41a7a744904b9759529a, State: Initialized, Role: FOLLOWER
I20260809 07:32:25.265600 30806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb995981efc41a7a744904b9759529a" member_type: VOTER }
I20260809 07:32:25.266002 30806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:32:25.266183 30806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:32:25.266444 30806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:25.270279 30806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb995981efc41a7a744904b9759529a" member_type: VOTER }
I20260809 07:32:25.270763 30806 leader_election.cc:302] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bb995981efc41a7a744904b9759529a; no voters:
I20260809 07:32:25.271790 30806 leader_election.cc:288] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:32:25.272157 30811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:25.273396 30811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [term 1 LEADER]: Becoming Leader. State: Replica: 8bb995981efc41a7a744904b9759529a, State: Running, Role: LEADER
I20260809 07:32:25.274041 30811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb995981efc41a7a744904b9759529a" member_type: VOTER }
I20260809 07:32:25.274590 30806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:32:25.278142 30812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bb995981efc41a7a744904b9759529a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb995981efc41a7a744904b9759529a" member_type: VOTER } }
I20260809 07:32:25.278481 30815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bb995981efc41a7a744904b9759529a. Latest consensus state: current_term: 1 leader_uuid: "8bb995981efc41a7a744904b9759529a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bb995981efc41a7a744904b9759529a" member_type: VOTER } }
I20260809 07:32:25.278803 30812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [sys.catalog]: This master's current role is: LEADER
I20260809 07:32:25.279189 30815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a [sys.catalog]: This master's current role is: LEADER
I20260809 07:32:25.281909 30824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:32:25.290228 30824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:32:25.292105 29021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260809 07:32:25.298962 30824 catalog_manager.cc:1347] Generated new cluster ID: 134a878f2e4f431aa9ae7c43981cd6c9
I20260809 07:32:25.299202 30824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:32:25.318858 30824 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:32:25.320937 30824 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:32:25.333515 30824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Generated new TSK 0
I20260809 07:32:25.334129 30824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:32:25.358492 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:32:25.364467 30837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.365437 30839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.366890 30842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:32:25.367275 29021 server_base.cc:1034] running on GCE node
I20260809 07:32:25.368283 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:32:25.368463 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:32:25.368626 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260745368611 us; error 0 us; skew 500 ppm
I20260809 07:32:25.369132 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:32:25.371415 29021 webserver.cc:460] Webserver started at http://127.28.87.65:44775/ using document root <none> and password file <none>
I20260809 07:32:25.371881 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:32:25.372041 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:32:25.372291 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:32:25.373332 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-0-root/instance:
uuid: "2c9de56c3fa3487c9dac990675c34be5"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.377749 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:32:25.380889 30847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.381659 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:32:25.381934 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-0-root
uuid: "2c9de56c3fa3487c9dac990675c34be5"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.382208 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:32:25.420243 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:32:25.421506 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:32:25.423087 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:32:25.425411 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:32:25.425603 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.425823 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:32:25.425982 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.465718 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.65:33749
I20260809 07:32:25.465821 30976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.65:33749 every 8 connection(s)
I20260809 07:32:25.470579 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:32:25.478523 30982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.479641 30987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.481148 30990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:32:25.484095 29021 server_base.cc:1034] running on GCE node
I20260809 07:32:25.485181 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:32:25.485416 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:32:25.485597 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260745485581 us; error 0 us; skew 500 ppm
I20260809 07:32:25.486135 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:32:25.486538 30977 heartbeater.cc:347] Connected to a master server at 127.28.87.126:41365
I20260809 07:32:25.486964 30977 heartbeater.cc:464] Registering TS with master...
I20260809 07:32:25.487927 30977 heartbeater.cc:511] Master 127.28.87.126:41365 requested a full tablet report, sending...
I20260809 07:32:25.489017 29021 webserver.cc:460] Webserver started at http://127.28.87.66:40573/ using document root <none> and password file <none>
I20260809 07:32:25.489584 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:32:25.489825 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:32:25.490150 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:32:25.490368 30745 ts_manager.cc:194] Registered new tserver with Master: 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:32:25.491899 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-1-root/instance:
uuid: "a720817fc113446a8ed55e61523b280a"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.493156 30745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40564
I20260809 07:32:25.497830 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 07:32:25.500993 31002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.501685 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:32:25.501965 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-1-root
uuid: "a720817fc113446a8ed55e61523b280a"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.502220 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:32:25.523733 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:32:25.524950 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:32:25.526501 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:32:25.528731 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:32:25.528936 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.529172 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:32:25.529331 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.568534 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.66:36075
I20260809 07:32:25.568636 31100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.66:36075 every 8 connection(s)
I20260809 07:32:25.573149 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:32:25.581297 31105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:25.583326 31106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:32:25.585819 29021 server_base.cc:1034] running on GCE node
W20260809 07:32:25.586149 31108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:32:25.587153 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260809 07:32:25.587342 31101 heartbeater.cc:347] Connected to a master server at 127.28.87.126:41365
W20260809 07:32:25.587469 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:32:25.587687 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260745587669 us; error 0 us; skew 500 ppm
I20260809 07:32:25.587723 31101 heartbeater.cc:464] Registering TS with master...
I20260809 07:32:25.588276 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:32:25.588428 31101 heartbeater.cc:511] Master 127.28.87.126:41365 requested a full tablet report, sending...
I20260809 07:32:25.590274 30745 ts_manager.cc:194] Registered new tserver with Master: a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:25.591017 29021 webserver.cc:460] Webserver started at http://127.28.87.67:36549/ using document root <none> and password file <none>
I20260809 07:32:25.591512 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:32:25.591699 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:32:25.591958 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:32:25.591996 30745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40566
I20260809 07:32:25.593371 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-2-root/instance:
uuid: "12107e38eb6d424abc4f45f16a9a1560"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.597621 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 07:32:25.600888 31116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.601584 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:32:25.601862 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-2-root
uuid: "12107e38eb6d424abc4f45f16a9a1560"
format_stamp: "Formatted at 2026-08-09 07:32:25 on dist-test-slave-fgb3"
I20260809 07:32:25.602123 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:32:25.612176 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:32:25.613325 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:32:25.614758 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:32:25.616868 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:32:25.617053 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.617267 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:32:25.617415 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:25.655169 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.67:44107
I20260809 07:32:25.655305 31219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.67:44107 every 8 connection(s)
I20260809 07:32:25.669518 31220 heartbeater.cc:347] Connected to a master server at 127.28.87.126:41365
I20260809 07:32:25.669888 31220 heartbeater.cc:464] Registering TS with master...
I20260809 07:32:25.670579 31220 heartbeater.cc:511] Master 127.28.87.126:41365 requested a full tablet report, sending...
I20260809 07:32:25.672492 30745 ts_manager.cc:194] Registered new tserver with Master: 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:25.672590 29021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014471582s
I20260809 07:32:25.674139 30745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40582
I20260809 07:32:25.674798 29021 test_util.cc:268] Using random seed: 1027137564
I20260809 07:32:25.698671 30745 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40584:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:32:25.704970 30745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:32:25.943004 31039 tablet_service.cc:1467] Processing CreateTablet for tablet dd3163efce7d454fb409a007f6ee93ef (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260809 07:32:25.946121 31029 tablet_service.cc:1467] Processing CreateTablet for tablet 24dd4a4091f24d51a4fae9e8d4ebca9e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260809 07:32:25.948567 31038 tablet_service.cc:1467] Processing CreateTablet for tablet e160777f9c924323b0223db59c1ae041 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260809 07:32:25.950989 31037 tablet_service.cc:1467] Processing CreateTablet for tablet cfb2fb324ee14e2084a95783b59cab00 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260809 07:32:25.957865 31151 tablet_service.cc:1467] Processing CreateTablet for tablet ee4aa83021af4237806656803bff3e10 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260809 07:32:25.962206 31160 tablet_service.cc:1467] Processing CreateTablet for tablet dd3163efce7d454fb409a007f6ee93ef (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260809 07:32:25.963229 31147 tablet_service.cc:1467] Processing CreateTablet for tablet 0a72c5de04d44d51b1e44b8be3e471b2 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260809 07:32:25.955428 31032 tablet_service.cc:1467] Processing CreateTablet for tablet ee4aa83021af4237806656803bff3e10 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260809 07:32:25.966180 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee4aa83021af4237806656803bff3e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.954200 31033 tablet_service.cc:1467] Processing CreateTablet for tablet 43645669985b45a0b7e6d18147b28424 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260809 07:32:25.967090 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a72c5de04d44d51b1e44b8be3e471b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.951478 31161 tablet_service.cc:1467] Processing CreateTablet for tablet 335fccb907184174b4a6428ce262e7dc (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260809 07:32:25.954201 31158 tablet_service.cc:1467] Processing CreateTablet for tablet cfb2fb324ee14e2084a95783b59cab00 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260809 07:32:25.968703 31161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fccb907184174b4a6428ce262e7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.943003 31040 tablet_service.cc:1467] Processing CreateTablet for tablet 335fccb907184174b4a6428ce262e7dc (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260809 07:32:25.959235 31146 tablet_service.cc:1467] Processing CreateTablet for tablet 24dd4a4091f24d51a4fae9e8d4ebca9e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260809 07:32:25.956641 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 43645669985b45a0b7e6d18147b28424 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260809 07:32:25.971256 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfb2fb324ee14e2084a95783b59cab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.952962 31159 tablet_service.cc:1467] Processing CreateTablet for tablet e160777f9c924323b0223db59c1ae041 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260809 07:32:25.970630 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd3163efce7d454fb409a007f6ee93ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.972921 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd3163efce7d454fb409a007f6ee93ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.963327 31148 tablet_service.cc:1467] Processing CreateTablet for tablet 3c486ed5f0d9423992400dfce71ce70f (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260809 07:32:25.974982 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24dd4a4091f24d51a4fae9e8d4ebca9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.975668 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c486ed5f0d9423992400dfce71ce70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.955428 31155 tablet_service.cc:1467] Processing CreateTablet for tablet 5df12775e0504b2da44e2fa37c5408ce (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260809 07:32:25.978778 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e160777f9c924323b0223db59c1ae041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.979385 31155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df12775e0504b2da44e2fa37c5408ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.980010 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43645669985b45a0b7e6d18147b28424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.974004 31029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24dd4a4091f24d51a4fae9e8d4ebca9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.982632 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfb2fb324ee14e2084a95783b59cab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.957872 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 0a72c5de04d44d51b1e44b8be3e471b2 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260809 07:32:25.956660 31031 tablet_service.cc:1467] Processing CreateTablet for tablet 3c486ed5f0d9423992400dfce71ce70f (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260809 07:32:25.985278 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e160777f9c924323b0223db59c1ae041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.952968 31034 tablet_service.cc:1467] Processing CreateTablet for tablet 5df12775e0504b2da44e2fa37c5408ce (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260809 07:32:25.987525 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43645669985b45a0b7e6d18147b28424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.988493 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c486ed5f0d9423992400dfce71ce70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.989230 31032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee4aa83021af4237806656803bff3e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.989825 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a72c5de04d44d51b1e44b8be3e471b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.991134 31040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fccb907184174b4a6428ce262e7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:25.991704 31034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df12775e0504b2da44e2fa37c5408ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.024824 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 87ac258020ac4e29b08523647d3ab361 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260809 07:32:26.044979 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ac258020ac4e29b08523647d3ab361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.054518 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 66e26e3128404ae8824fab16033fc0ea (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260809 07:32:26.055552 30889 tablet_service.cc:1467] Processing CreateTablet for tablet 335fccb907184174b4a6428ce262e7dc (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260809 07:32:26.056423 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e26e3128404ae8824fab16033fc0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.056931 30889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fccb907184174b4a6428ce262e7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.059531 30885 tablet_service.cc:1467] Processing CreateTablet for tablet dd3163efce7d454fb409a007f6ee93ef (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260809 07:32:26.060768 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd3163efce7d454fb409a007f6ee93ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.060477 31159 tablet_service.cc:1467] Processing CreateTablet for tablet 4930be41b06547ba93e04e5d1385cb3c (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260809 07:32:26.061655 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4930be41b06547ba93e04e5d1385cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.063498 30884 tablet_service.cc:1467] Processing CreateTablet for tablet e160777f9c924323b0223db59c1ae041 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260809 07:32:26.064796 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e160777f9c924323b0223db59c1ae041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.067469 30883 tablet_service.cc:1467] Processing CreateTablet for tablet cfb2fb324ee14e2084a95783b59cab00 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260809 07:32:26.068686 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfb2fb324ee14e2084a95783b59cab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.068974 31161 tablet_service.cc:1467] Processing CreateTablet for tablet 9dba78a1408b4bd7a6f0c247b1176f0b (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260809 07:32:26.070075 31161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dba78a1408b4bd7a6f0c247b1176f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.072834 30882 tablet_service.cc:1467] Processing CreateTablet for tablet 5df12775e0504b2da44e2fa37c5408ce (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260809 07:32:26.075450 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df12775e0504b2da44e2fa37c5408ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.077807 31151 tablet_service.cc:1467] Processing CreateTablet for tablet b4960950e96549f499238d58397046ce (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260809 07:32:26.073486 30881 tablet_service.cc:1467] Processing CreateTablet for tablet 43645669985b45a0b7e6d18147b28424 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260809 07:32:26.079025 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4960950e96549f499238d58397046ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.079479 30880 tablet_service.cc:1467] Processing CreateTablet for tablet ee4aa83021af4237806656803bff3e10 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260809 07:32:26.081771 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 3c486ed5f0d9423992400dfce71ce70f (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260809 07:32:26.081998 31148 tablet_service.cc:1467] Processing CreateTablet for tablet beb03248d97e4a8c8ea50e8576f8cb6e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260809 07:32:26.083995 30877 tablet_service.cc:1467] Processing CreateTablet for tablet 0a72c5de04d44d51b1e44b8be3e471b2 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260809 07:32:26.085671 30875 tablet_service.cc:1467] Processing CreateTablet for tablet 24dd4a4091f24d51a4fae9e8d4ebca9e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260809 07:32:26.079396 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43645669985b45a0b7e6d18147b28424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.094462 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24dd4a4091f24d51a4fae9e8d4ebca9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.095573 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c486ed5f0d9423992400dfce71ce70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.098634 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a72c5de04d44d51b1e44b8be3e471b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.109056 30880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee4aa83021af4237806656803bff3e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.109889 31243 tablet_bootstrap.cc:492] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:26.110445 31034 tablet_service.cc:1467] Processing CreateTablet for tablet 87ac258020ac4e29b08523647d3ab361 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260809 07:32:26.111667 31034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ac258020ac4e29b08523647d3ab361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.114445 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb03248d97e4a8c8ea50e8576f8cb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.115635 31038 tablet_service.cc:1467] Processing CreateTablet for tablet 9dba78a1408b4bd7a6f0c247b1176f0b (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260809 07:32:26.116863 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dba78a1408b4bd7a6f0c247b1176f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.117411 31147 tablet_service.cc:1467] Processing CreateTablet for tablet 976fdb751bc0440a92f97b6c95314270 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260809 07:32:26.118610 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 976fdb751bc0440a92f97b6c95314270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.115550 31033 tablet_service.cc:1467] Processing CreateTablet for tablet 4930be41b06547ba93e04e5d1385cb3c (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260809 07:32:26.124189 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4930be41b06547ba93e04e5d1385cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.119553 31030 tablet_service.cc:1467] Processing CreateTablet for tablet b4960950e96549f499238d58397046ce (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260809 07:32:26.127048 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4960950e96549f499238d58397046ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.127693 31029 tablet_service.cc:1467] Processing CreateTablet for tablet fcad7c2c5a034837a995628cd23c8a8b (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260809 07:32:26.128948 31029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcad7c2c5a034837a995628cd23c8a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.113828 31040 tablet_service.cc:1467] Processing CreateTablet for tablet 66e26e3128404ae8824fab16033fc0ea (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260809 07:32:26.122831 31031 tablet_service.cc:1467] Processing CreateTablet for tablet beb03248d97e4a8c8ea50e8576f8cb6e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260809 07:32:26.135054 31040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e26e3128404ae8824fab16033fc0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.135653 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb03248d97e4a8c8ea50e8576f8cb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.135962 31037 tablet_service.cc:1467] Processing CreateTablet for tablet 976fdb751bc0440a92f97b6c95314270 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260809 07:32:26.137171 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 976fdb751bc0440a92f97b6c95314270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.141109 31155 tablet_service.cc:1467] Processing CreateTablet for tablet 7635c2a6de6446b294b707dbb8bf244a (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260809 07:32:26.145927 31243 tablet_bootstrap.cc:654] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:26.158017 31158 tablet_service.cc:1467] Processing CreateTablet for tablet fcad7c2c5a034837a995628cd23c8a8b (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260809 07:32:26.163341 31146 tablet_service.cc:1467] Processing CreateTablet for tablet 656029f31f2846908145dfacac4abdf2 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260809 07:32:26.168087 31039 tablet_service.cc:1467] Processing CreateTablet for tablet 7635c2a6de6446b294b707dbb8bf244a (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260809 07:32:26.169486 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7635c2a6de6446b294b707dbb8bf244a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.176837 31155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7635c2a6de6446b294b707dbb8bf244a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.179003 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcad7c2c5a034837a995628cd23c8a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.186796 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656029f31f2846908145dfacac4abdf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.201411 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 87ac258020ac4e29b08523647d3ab361 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260809 07:32:26.212091 30883 tablet_service.cc:1467] Processing CreateTablet for tablet 66e26e3128404ae8824fab16033fc0ea (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260809 07:32:26.213444 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e26e3128404ae8824fab16033fc0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.229152 31244 tablet_bootstrap.cc:492] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:26.231945 31032 tablet_service.cc:1467] Processing CreateTablet for tablet 656029f31f2846908145dfacac4abdf2 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260809 07:32:26.233230 31032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656029f31f2846908145dfacac4abdf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.235296 30884 tablet_service.cc:1467] Processing CreateTablet for tablet b4960950e96549f499238d58397046ce (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260809 07:32:26.237478 30875 tablet_service.cc:1467] Processing CreateTablet for tablet 4930be41b06547ba93e04e5d1385cb3c (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260809 07:32:26.237612 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 485e0cc914a748fa957f0291e4fea329 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260809 07:32:26.239880 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 485e0cc914a748fa957f0291e4fea329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.243528 31245 tablet_bootstrap.cc:492] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:26.245509 30877 tablet_service.cc:1467] Processing CreateTablet for tablet beb03248d97e4a8c8ea50e8576f8cb6e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260809 07:32:26.248976 30881 tablet_service.cc:1467] Processing CreateTablet for tablet fcad7c2c5a034837a995628cd23c8a8b (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260809 07:32:26.250397 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ac258020ac4e29b08523647d3ab361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.255438 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb9e791b482439b936398015bc42aff (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260809 07:32:26.256642 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb9e791b482439b936398015bc42aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.258432 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4960950e96549f499238d58397046ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.262598 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4930be41b06547ba93e04e5d1385cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.267530 31244 tablet_bootstrap.cc:654] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:26.272184 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 0e6264779cf74c14bc8c18901dfe6a09 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260809 07:32:26.273499 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e6264779cf74c14bc8c18901dfe6a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.279229 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 485e0cc914a748fa957f0291e4fea329 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
W20260809 07:32:26.288216 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:26.295172 30889 tablet_service.cc:1467] Processing CreateTablet for tablet 976fdb751bc0440a92f97b6c95314270 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260809 07:32:26.296697 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb03248d97e4a8c8ea50e8576f8cb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.298372 31245 tablet_bootstrap.cc:654] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:26.299427 31159 tablet_service.cc:1467] Processing CreateTablet for tablet 8e327743625143d2b7f941cececc7d75 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260809 07:32:26.300778 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e327743625143d2b7f941cececc7d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.301669 30889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 976fdb751bc0440a92f97b6c95314270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.304292 31151 tablet_service.cc:1467] Processing CreateTablet for tablet 5e48f193f20844a4ac619300cfaac1bd (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260809 07:32:26.305527 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f193f20844a4ac619300cfaac1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.320642 30880 tablet_service.cc:1467] Processing CreateTablet for tablet 9dba78a1408b4bd7a6f0c247b1176f0b (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260809 07:32:26.322091 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcad7c2c5a034837a995628cd23c8a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.330061 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 485e0cc914a748fa957f0291e4fea329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.345227 31155 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1ac5b8f9784e3b9c028fa7f7bdf1ea (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260809 07:32:26.352526 31146 tablet_service.cc:1467] Processing CreateTablet for tablet c1ccd6862a094ecba07b495d4c772721 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260809 07:32:26.354118 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ccd6862a094ecba07b495d4c772721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.325903 31148 tablet_service.cc:1467] Processing CreateTablet for tablet f25fc40767244ba0873588714d88e8b6 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260809 07:32:26.269974 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 401ec3203e7b44faae0c513339cc7317 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260809 07:32:26.356899 31147 tablet_service.cc:1467] Processing CreateTablet for tablet ca210f0f3a954ffb8a42764c24b02830 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260809 07:32:26.362403 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401ec3203e7b44faae0c513339cc7317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.366958 31245 tablet_bootstrap.cc:492] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:26.642490 31245 ts_tablet_manager.cc:1397] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.399s user 0.027s sys 0.025s
I20260809 07:32:26.644841 31245 raft_consensus.cc:348] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.365283 31243 tablet_bootstrap.cc:492] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:26.646714 31243 ts_tablet_manager.cc:1397] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.537s user 0.025s sys 0.000s
I20260809 07:32:26.499497 31029 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb9e791b482439b936398015bc42aff (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260809 07:32:26.509712 31031 tablet_service.cc:1467] Processing CreateTablet for tablet 0e6264779cf74c14bc8c18901dfe6a09 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260809 07:32:26.519594 31033 tablet_service.cc:1467] Processing CreateTablet for tablet 8e327743625143d2b7f941cececc7d75 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260809 07:32:26.290863 31038 tablet_service.cc:1467] Processing CreateTablet for tablet 5e48f193f20844a4ac619300cfaac1bd (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260809 07:32:26.542042 31151 tablet_service.cc:1467] Processing CreateTablet for tablet aeccd39240c34b5fabec551b21206a7f (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260809 07:32:26.648252 31029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb9e791b482439b936398015bc42aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.648926 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeccd39240c34b5fabec551b21206a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.649286 31243 raft_consensus.cc:348] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.589547 31152 tablet_service.cc:1467] Processing CreateTablet for tablet e580429091da41448485d00896fcf32e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260809 07:32:26.650018 31243 raft_consensus.cc:374] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:26.650679 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e580429091da41448485d00896fcf32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.606739 31158 tablet_service.cc:1467] Processing CreateTablet for tablet 43743864250e4199a5862ae02b5dbed7 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260809 07:32:26.573854 31039 tablet_service.cc:1467] Processing CreateTablet for tablet e580429091da41448485d00896fcf32e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260809 07:32:26.618012 31034 tablet_service.cc:1467] Processing CreateTablet for tablet afa0c0ea2e5044208ed578990bfd5d37 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260809 07:32:26.618430 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1ac5b8f9784e3b9c028fa7f7bdf1ea (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260809 07:32:26.639070 31101 heartbeater.cc:503] Master 127.28.87.126:41365 was elected leader, sending a full tablet report...
I20260809 07:32:26.657362 31159 tablet_service.cc:1467] Processing CreateTablet for tablet 17576a737982459aa79301ce4f7eb096 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260809 07:32:26.660164 31161 tablet_service.cc:1467] Processing CreateTablet for tablet afa0c0ea2e5044208ed578990bfd5d37 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260809 07:32:26.662158 31040 tablet_service.cc:1467] Processing CreateTablet for tablet 401ec3203e7b44faae0c513339cc7317 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260809 07:32:26.615973 31032 tablet_service.cc:1467] Processing CreateTablet for tablet f25fc40767244ba0873588714d88e8b6 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260809 07:32:26.648900 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e327743625143d2b7f941cececc7d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.666868 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1ac5b8f9784e3b9c028fa7f7bdf1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.597790 30977 heartbeater.cc:503] Master 127.28.87.126:41365 was elected leader, sending a full tablet report...
I20260809 07:32:26.667824 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e580429091da41448485d00896fcf32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.655143 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43743864250e4199a5862ae02b5dbed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.669001 31032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25fc40767244ba0873588714d88e8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.670661 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca210f0f3a954ffb8a42764c24b02830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.672278 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e6264779cf74c14bc8c18901dfe6a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.672394 31155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1ac5b8f9784e3b9c028fa7f7bdf1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.672784 31037 tablet_service.cc:1467] Processing CreateTablet for tablet c1ccd6862a094ecba07b495d4c772721 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260809 07:32:26.673988 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ccd6862a094ecba07b495d4c772721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.674572 31034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa0c0ea2e5044208ed578990bfd5d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.677258 31220 heartbeater.cc:503] Master 127.28.87.126:41365 was elected leader, sending a full tablet report...
I20260809 07:32:26.677732 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f193f20844a4ac619300cfaac1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.679271 31040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401ec3203e7b44faae0c513339cc7317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.677171 31244 tablet_bootstrap.cc:492] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:26.680270 31244 ts_tablet_manager.cc:1397] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.451s user 0.008s sys 0.023s
I20260809 07:32:26.655401 31243 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:26.683240 31243 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.686220 31244 raft_consensus.cc:348] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.687502 31244 raft_consensus.cc:374] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:26.687778 31244 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:26.687721 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25fc40767244ba0873588714d88e8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.645793 31245 raft_consensus.cc:374] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:26.690452 31245 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:26.691155 31245 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.693212 30883 tablet_service.cc:1467] Processing CreateTablet for tablet e580429091da41448485d00896fcf32e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260809 07:32:26.695691 30882 tablet_service.cc:1467] Processing CreateTablet for tablet 7635c2a6de6446b294b707dbb8bf244a (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260809 07:32:26.698424 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e580429091da41448485d00896fcf32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.700093 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17576a737982459aa79301ce4f7eb096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.700798 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7635c2a6de6446b294b707dbb8bf244a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.701076 30877 tablet_service.cc:1467] Processing CreateTablet for tablet 656029f31f2846908145dfacac4abdf2 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260809 07:32:26.702281 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656029f31f2846908145dfacac4abdf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.704895 30884 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1ac5b8f9784e3b9c028fa7f7bdf1ea (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260809 07:32:26.706061 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1ac5b8f9784e3b9c028fa7f7bdf1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.706168 30881 tablet_service.cc:1467] Processing CreateTablet for tablet 5e48f193f20844a4ac619300cfaac1bd (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260809 07:32:26.712078 31037 tablet_service.cc:1467] Processing CreateTablet for tablet ca210f0f3a954ffb8a42764c24b02830 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260809 07:32:26.704895 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 485e0cc914a748fa957f0291e4fea329 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260809 07:32:26.716064 30875 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb9e791b482439b936398015bc42aff (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260809 07:32:26.729621 31158 tablet_service.cc:1467] Processing CreateTablet for tablet 08675f2b92e541ecba618784caa19bb3 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260809 07:32:26.718008 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 8e327743625143d2b7f941cececc7d75 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260809 07:32:26.739077 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb9e791b482439b936398015bc42aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.722126 31161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa0c0ea2e5044208ed578990bfd5d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.740392 30880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dba78a1408b4bd7a6f0c247b1176f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.741986 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e48f193f20844a4ac619300cfaac1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.746595 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e327743625143d2b7f941cececc7d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.747259 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 485e0cc914a748fa957f0291e4fea329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.718135 30889 tablet_service.cc:1467] Processing CreateTablet for tablet 401ec3203e7b44faae0c513339cc7317 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260809 07:32:26.762707 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08675f2b92e541ecba618784caa19bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.763231 30889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401ec3203e7b44faae0c513339cc7317. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 07:32:26.770799 30724 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46509.9us
I20260809 07:32:26.774904 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca210f0f3a954ffb8a42764c24b02830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.776515 31030 tablet_service.cc:1467] Processing CreateTablet for tablet aeccd39240c34b5fabec551b21206a7f (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260809 07:32:26.777761 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeccd39240c34b5fabec551b21206a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.795282 31245 ts_tablet_manager.cc:1428] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.152s user 0.007s sys 0.000s
I20260809 07:32:26.796301 31245 tablet_bootstrap.cc:492] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:26.798116 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 964c4cb7bad240cca6d84d160e658ffe (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260809 07:32:26.799489 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 964c4cb7bad240cca6d84d160e658ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.802307 31147 tablet_service.cc:1467] Processing CreateTablet for tablet b2be7407d0cf4ac2a1a0a1e6525c9bac (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260809 07:32:26.804499 31039 tablet_service.cc:1467] Processing CreateTablet for tablet 17576a737982459aa79301ce4f7eb096 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260809 07:32:26.811306 30875 tablet_service.cc:1467] Processing CreateTablet for tablet 0e6264779cf74c14bc8c18901dfe6a09 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260809 07:32:26.722254 31244 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.806229 31245 tablet_bootstrap.cc:654] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:26.839636 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17576a737982459aa79301ce4f7eb096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.818945 31033 tablet_service.cc:1467] Processing CreateTablet for tablet 964c4cb7bad240cca6d84d160e658ffe (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260809 07:32:26.842582 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2be7407d0cf4ac2a1a0a1e6525c9bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.843169 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 964c4cb7bad240cca6d84d160e658ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.847981 31029 tablet_service.cc:1467] Processing CreateTablet for tablet e8bb7f31be3d4e2faf59b0f97f6a803d (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260809 07:32:26.849262 31029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8bb7f31be3d4e2faf59b0f97f6a803d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.852262 31243 ts_tablet_manager.cc:1428] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.205s user 0.015s sys 0.046s
I20260809 07:32:26.823990 31151 tablet_service.cc:1467] Processing CreateTablet for tablet e8bb7f31be3d4e2faf59b0f97f6a803d (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260809 07:32:26.855393 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8bb7f31be3d4e2faf59b0f97f6a803d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.860715 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 11936edbc86a4c4fbf8e08748a48d8cb (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260809 07:32:26.861986 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11936edbc86a4c4fbf8e08748a48d8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.863085 31243 tablet_bootstrap.cc:492] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:26.825961 31158 tablet_service.cc:1467] Processing CreateTablet for tablet f4d4e67ddf074e90a2d36bc4e649b5c1 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260809 07:32:26.817867 31032 tablet_service.cc:1467] Processing CreateTablet for tablet 08675f2b92e541ecba618784caa19bb3 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260809 07:32:26.868165 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d4e67ddf074e90a2d36bc4e649b5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.865828 31031 tablet_service.cc:1467] Processing CreateTablet for tablet f4d4e67ddf074e90a2d36bc4e649b5c1 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260809 07:32:26.820808 31040 tablet_service.cc:1467] Processing CreateTablet for tablet b2be7407d0cf4ac2a1a0a1e6525c9bac (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260809 07:32:26.868160 31032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08675f2b92e541ecba618784caa19bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.870779 31040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2be7407d0cf4ac2a1a0a1e6525c9bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.872318 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e6264779cf74c14bc8c18901dfe6a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.872648 31148 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed1309ae2b34b6085b236210555d243 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260809 07:32:26.873749 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed1309ae2b34b6085b236210555d243. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 07:32:26.875970 30724 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 58613.8us
W20260809 07:32:26.876253 30725 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 50913.1us
I20260809 07:32:26.865994 31034 tablet_service.cc:1467] Processing CreateTablet for tablet 43743864250e4199a5862ae02b5dbed7 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260809 07:32:26.893540 30885 tablet_service.cc:1467] Processing CreateTablet for tablet c1ccd6862a094ecba07b495d4c772721 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260809 07:32:26.893829 31038 tablet_service.cc:1467] Processing CreateTablet for tablet 11936edbc86a4c4fbf8e08748a48d8cb (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260809 07:32:26.869562 30877 tablet_service.cc:1467] Processing CreateTablet for tablet afa0c0ea2e5044208ed578990bfd5d37 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260809 07:32:26.886528 30883 tablet_service.cc:1467] Processing CreateTablet for tablet f25fc40767244ba0873588714d88e8b6 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260809 07:32:26.865031 31155 tablet_service.cc:1467] Processing CreateTablet for tablet 5b57d0ed4ced43cfbe36695b39b05897 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260809 07:32:26.910629 31155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b57d0ed4ced43cfbe36695b39b05897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.913471 30881 tablet_service.cc:1467] Processing CreateTablet for tablet ca210f0f3a954ffb8a42764c24b02830 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260809 07:32:26.886941 31146 tablet_service.cc:1467] Processing CreateTablet for tablet 202d19fadaa547209afb8e0109ff54e8 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260809 07:32:26.914808 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 202d19fadaa547209afb8e0109ff54e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.914853 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ccd6862a094ecba07b495d4c772721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.877012 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d4e67ddf074e90a2d36bc4e649b5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.923434 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 5b57d0ed4ced43cfbe36695b39b05897 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260809 07:32:26.924701 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b57d0ed4ced43cfbe36695b39b05897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.928525 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa0c0ea2e5044208ed578990bfd5d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.936232 31031 tablet_service.cc:1467] Processing CreateTablet for tablet 202d19fadaa547209afb8e0109ff54e8 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260809 07:32:26.938580 31034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43743864250e4199a5862ae02b5dbed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.939183 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11936edbc86a4c4fbf8e08748a48d8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.940761 30889 tablet_service.cc:1467] Processing CreateTablet for tablet aeccd39240c34b5fabec551b21206a7f (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260809 07:32:26.941525 30880 tablet_service.cc:1467] Processing CreateTablet for tablet 43743864250e4199a5862ae02b5dbed7 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260809 07:32:26.941952 30889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeccd39240c34b5fabec551b21206a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.945417 30884 tablet_service.cc:1467] Processing CreateTablet for tablet 17576a737982459aa79301ce4f7eb096 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260809 07:32:26.946149 30882 tablet_service.cc:1467] Processing CreateTablet for tablet e8bb7f31be3d4e2faf59b0f97f6a803d (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260809 07:32:26.942685 30880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43743864250e4199a5862ae02b5dbed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.951016 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25fc40767244ba0873588714d88e8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.952179 31158 tablet_service.cc:1467] Processing CreateTablet for tablet 5f1326c9441f41f4ad3abaac9ca2b2ec (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260809 07:32:26.953336 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f1326c9441f41f4ad3abaac9ca2b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.953569 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17576a737982459aa79301ce4f7eb096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.950935 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 202d19fadaa547209afb8e0109ff54e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.937098 31037 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed1309ae2b34b6085b236210555d243 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260809 07:32:26.962491 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca210f0f3a954ffb8a42764c24b02830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.963119 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed1309ae2b34b6085b236210555d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.964277 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8bb7f31be3d4e2faf59b0f97f6a803d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.967149 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 08675f2b92e541ecba618784caa19bb3 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260809 07:32:26.968322 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08675f2b92e541ecba618784caa19bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.970990 31244 ts_tablet_manager.cc:1428] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.290s user 0.012s sys 0.004s
I20260809 07:32:26.974551 31244 tablet_bootstrap.cc:492] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:26.975237 31161 tablet_service.cc:1467] Processing CreateTablet for tablet ae547589201f452eb2ae83ea11689f7e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260809 07:32:26.975901 31253 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:26.976742 31161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae547589201f452eb2ae83ea11689f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.976459 30884 tablet_service.cc:1467] Processing CreateTablet for tablet 964c4cb7bad240cca6d84d160e658ffe (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260809 07:32:26.976418 31253 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:26.987217 31253 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:26.991514 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 964c4cb7bad240cca6d84d160e658ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:26.993449 31159 tablet_service.cc:1467] Processing CreateTablet for tablet 9e87f17b17dd4e1fba73ed247d6b4039 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260809 07:32:26.994728 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e87f17b17dd4e1fba73ed247d6b4039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.002868 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 4d8f75ef38314f3185fbcd1e0b1df988 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260809 07:32:27.007625 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d8f75ef38314f3185fbcd1e0b1df988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.029538 31032 tablet_service.cc:1467] Processing CreateTablet for tablet 5f1326c9441f41f4ad3abaac9ca2b2ec (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260809 07:32:27.030875 31032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f1326c9441f41f4ad3abaac9ca2b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.048946 31147 tablet_service.cc:1467] Processing CreateTablet for tablet 51e40c75108c4a86a26730860edee159 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260809 07:32:27.050908 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51e40c75108c4a86a26730860edee159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.053193 31243 tablet_bootstrap.cc:654] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.058521 31151 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe29d891e5e411e8c46c18dcae14e66 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260809 07:32:27.059423 31040 tablet_service.cc:1467] Processing CreateTablet for tablet ae547589201f452eb2ae83ea11689f7e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260809 07:32:27.059849 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe29d891e5e411e8c46c18dcae14e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.060626 31040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae547589201f452eb2ae83ea11689f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.062716 31039 tablet_service.cc:1467] Processing CreateTablet for tablet 4d8f75ef38314f3185fbcd1e0b1df988 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260809 07:32:27.070008 31245 tablet_bootstrap.cc:492] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.070557 31245 ts_tablet_manager.cc:1397] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.274s user 0.013s sys 0.008s
I20260809 07:32:27.073160 31245 raft_consensus.cc:348] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.073834 31245 raft_consensus.cc:374] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.070557 31033 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe29d891e5e411e8c46c18dcae14e66 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260809 07:32:27.077888 30882 tablet_service.cc:1467] Processing CreateTablet for tablet f4d4e67ddf074e90a2d36bc4e649b5c1 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260809 07:32:27.083549 31160 tablet_service.cc:1467] Processing CreateTablet for tablet 59bd1fdaac4346aa9f2f9be0f904b408 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260809 07:32:27.086125 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59bd1fdaac4346aa9f2f9be0f904b408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.084105 31146 tablet_service.cc:1467] Processing CreateTablet for tablet b72ff16aedce4676b889395ca4d8909d (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260809 07:32:27.074116 31245 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.087011 30875 tablet_service.cc:1467] Processing CreateTablet for tablet 11936edbc86a4c4fbf8e08748a48d8cb (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260809 07:32:27.088272 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72ff16aedce4676b889395ca4d8909d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.087915 31245 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.092589 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d4e67ddf074e90a2d36bc4e649b5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.083808 31152 tablet_service.cc:1467] Processing CreateTablet for tablet 723b3b7ed7b749d3b4b448d22577a8e7 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260809 07:32:27.103617 31152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 723b3b7ed7b749d3b4b448d22577a8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.105706 31244 tablet_bootstrap.cc:654] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.109669 31254 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.110139 31254 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.117329 31254 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.067734 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d8f75ef38314f3185fbcd1e0b1df988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.114557 31245 ts_tablet_manager.cc:1428] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260809 07:32:27.127650 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11936edbc86a4c4fbf8e08748a48d8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.128026 31245 tablet_bootstrap.cc:492] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:27.146124 31148 tablet_service.cc:1467] Processing CreateTablet for tablet 3bbfdffc27ac46a0a18582f85cd4a00a (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260809 07:32:27.147405 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbfdffc27ac46a0a18582f85cd4a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.148195 31252 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.148679 31252 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.148904 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 51e40c75108c4a86a26730860edee159 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260809 07:32:27.132884 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe29d891e5e411e8c46c18dcae14e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.156383 30877 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 27) took 1064 ms. Trace:
I20260809 07:32:27.156495 31252 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:27.190677 30883 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 28) took 1098 ms. Trace:
I20260809 07:32:27.191002 31032 tablet_service.cc:1467] Processing CreateTablet for tablet 59bd1fdaac4346aa9f2f9be0f904b408 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260809 07:32:27.191152 30883 rpcz_store.cc:276] 0809 07:32:26.092113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.885628 (+793515us) service_pool.cc:227] Handling call
0809 07:32:26.943886 (+ 58258us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.943993 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.003471 (+ 59478us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.157674 (+154203us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.190603 (+ 32929us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.193755 30883 tablet_service.cc:1467] Processing CreateTablet for tablet 5b57d0ed4ced43cfbe36695b39b05897 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260809 07:32:27.196394 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b57d0ed4ced43cfbe36695b39b05897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.203409 30880 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 33) took 1006 ms. Trace:
I20260809 07:32:27.203910 30880 rpcz_store.cc:276] 0809 07:32:26.202288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.940725 (+738437us) service_pool.cc:227] Handling call
0809 07:32:26.942253 (+ 1528us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.942374 (+ 121us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.055924 (+113550us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.202841 (+146917us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.203340 (+ 499us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.205925 30880 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed1309ae2b34b6085b236210555d243 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260809 07:32:27.224146 30880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed1309ae2b34b6085b236210555d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.162420 30877 rpcz_store.cc:276] 0809 07:32:26.091796 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.868757 (+776961us) service_pool.cc:227] Handling call
0809 07:32:26.926999 (+ 58242us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.928164 (+ 1165us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:26.978479 (+ 50315us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.155890 (+177411us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.156324 (+ 434us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1826048}
I20260809 07:32:27.213104 31155 tablet_service.cc:1467] Processing CreateTablet for tablet 1a18497352404d2491d49a9241fe61bf (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260809 07:32:27.168763 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51e40c75108c4a86a26730860edee159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.146139 31031 tablet_service.cc:1467] Processing CreateTablet for tablet 9e87f17b17dd4e1fba73ed247d6b4039 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260809 07:32:27.219807 31245 tablet_bootstrap.cc:654] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.221151 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 202d19fadaa547209afb8e0109ff54e8 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260809 07:32:27.223042 30881 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 31) took 1130 ms. Trace:
I20260809 07:32:27.237075 31161 tablet_service.cc:1467] Processing CreateTablet for tablet 31954ed4d71d4a6796aa3c4bb2ded761 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260809 07:32:27.179786 30885 tablet_service.cc:1467] Processing CreateTablet for tablet b2be7407d0cf4ac2a1a0a1e6525c9bac (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260809 07:32:27.211174 30889 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 32) took 1022 ms. Trace:
I20260809 07:32:27.253096 31155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a18497352404d2491d49a9241fe61bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.253688 31032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59bd1fdaac4346aa9f2f9be0f904b408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.254896 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 202d19fadaa547209afb8e0109ff54e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.266744 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2be7407d0cf4ac2a1a0a1e6525c9bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.272101 31037 tablet_service.cc:1467] Processing CreateTablet for tablet b72ff16aedce4676b889395ca4d8909d (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260809 07:32:27.249980 30889 rpcz_store.cc:276] 0809 07:32:26.188608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.939981 (+751373us) service_pool.cc:227] Handling call
0809 07:32:26.941518 (+ 1537us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.941609 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.054752 (+113143us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.210663 (+155911us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.211111 (+ 448us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.279341 31148 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba1c2ba04d7459b93ff369f6223cef6 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260809 07:32:27.280435 30889 tablet_service.cc:1467] Processing CreateTablet for tablet ae547589201f452eb2ae83ea11689f7e (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260809 07:32:27.247347 30881 rpcz_store.cc:276] 0809 07:32:26.093076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.912584 (+819508us) service_pool.cc:227] Handling call
0809 07:32:26.946296 (+ 33712us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.946513 (+ 217us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.039131 (+ 92618us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.222515 (+183384us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.222978 (+ 463us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.270217 31031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e87f17b17dd4e1fba73ed247d6b4039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.286623 30881 tablet_service.cc:1467] Processing CreateTablet for tablet 4d8f75ef38314f3185fbcd1e0b1df988 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260809 07:32:27.287591 30889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae547589201f452eb2ae83ea11689f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.288278 31039 tablet_service.cc:1467] Processing CreateTablet for tablet 723b3b7ed7b749d3b4b448d22577a8e7 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260809 07:32:27.313789 31146 tablet_service.cc:1467] Processing CreateTablet for tablet cb8db10c4bb04e86a15bcbff5ffed3d5 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260809 07:32:27.273666 31161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31954ed4d71d4a6796aa3c4bb2ded761. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 07:32:27.296353 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:27.299439 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.303618 31244 tablet_bootstrap.cc:492] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.304072 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72ff16aedce4676b889395ca4d8909d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.306993 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:27.324139 30877 tablet_service.cc:1467] Processing CreateTablet for tablet 5f1326c9441f41f4ad3abaac9ca2b2ec (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260809 07:32:27.327320 31243 tablet_bootstrap.cc:492] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.327790 31243 ts_tablet_manager.cc:1397] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.465s user 0.038s sys 0.047s
I20260809 07:32:27.308759 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:27.331305 31006 leader_election.cc:341] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.331874 31244 ts_tablet_manager.cc:1397] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.358s user 0.012s sys 0.028s
I20260809 07:32:27.332201 31034 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36476 (request call id 33) took 1205 ms. Trace:
I20260809 07:32:27.332597 31034 rpcz_store.cc:276] 0809 07:32:26.126852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.809922 (+683070us) service_pool.cc:227] Handling call
0809 07:32:26.880942 (+ 71020us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.881036 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.020115 (+139079us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.331673 (+311558us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.332137 (+ 464us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.334506 31244 raft_consensus.cc:348] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.335245 31244 raft_consensus.cc:374] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.335548 31244 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.276647 31159 tablet_service.cc:1467] Processing CreateTablet for tablet c2c9e03f3fb24c0eba0bf8f658070202 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260809 07:32:27.321529 30853 leader_election.cc:341] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:27.337009 30852 leader_election.cc:341] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.339218 30852 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:27.322115 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba1c2ba04d7459b93ff369f6223cef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.340006 31253 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.342104 30884 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 36) took 1114 ms. Trace:
I20260809 07:32:27.326304 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 723b3b7ed7b749d3b4b448d22577a8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.342518 30884 rpcz_store.cc:276] 0809 07:32:26.228681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.974090 (+745409us) service_pool.cc:227] Handling call
0809 07:32:26.977378 (+ 3288us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.977468 (+ 90us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.132408 (+154940us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.341566 (+209158us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.342045 (+ 479us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.344707 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d8f75ef38314f3185fbcd1e0b1df988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.344444 30884 tablet_service.cc:1467] Processing CreateTablet for tablet 0fe29d891e5e411e8c46c18dcae14e66 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260809 07:32:27.345666 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fe29d891e5e411e8c46c18dcae14e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.347851 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:27.346628 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c9e03f3fb24c0eba0bf8f658070202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.349464 31030 tablet_service.cc:1467] Processing CreateTablet for tablet 3bbfdffc27ac46a0a18582f85cd4a00a (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260809 07:32:27.353720 31005 leader_election.cc:341] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.354177 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.317597 31029 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36476 (request call id 38) took 1112 ms. Trace:
I20260809 07:32:27.355032 31029 rpcz_store.cc:276] 0809 07:32:26.203470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.847070 (+643600us) service_pool.cc:227] Handling call
0809 07:32:26.848829 (+ 1759us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.848919 (+ 90us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:26.986916 (+137997us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.262939 (+276023us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.315755 (+ 52816us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.355458 31254 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.356505 31243 raft_consensus.cc:348] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.357229 31243 raft_consensus.cc:374] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.357527 31243 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.358210 31029 tablet_service.cc:1467] Processing CreateTablet for tablet 1a18497352404d2491d49a9241fe61bf (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260809 07:32:27.362108 30880 tablet_service.cc:1467] Processing CreateTablet for tablet 9e87f17b17dd4e1fba73ed247d6b4039 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260809 07:32:27.363623 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbfdffc27ac46a0a18582f85cd4a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.366506 31146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8db10c4bb04e86a15bcbff5ffed3d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.366150 31243 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.370853 31243 ts_tablet_manager.cc:1428] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.043s user 0.006s sys 0.002s
I20260809 07:32:27.372570 31243 tablet_bootstrap.cc:492] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:27.371476 30885 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 37) took 1142 ms. Trace:
W20260809 07:32:27.374909 30796 debug-util.cc:398] Leaking SignalData structure 0x7b080008e1a0 after lost signal to thread 29026
W20260809 07:32:27.375833 30796 debug-util.cc:398] Leaking SignalData structure 0x7b080003b6a0 after lost signal to thread 30803
I20260809 07:32:27.376152 31040 tablet_service.cc:1467] Processing CreateTablet for tablet c2c9e03f3fb24c0eba0bf8f658070202 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260809 07:32:27.377710 31040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c9e03f3fb24c0eba0bf8f658070202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.348246 31038 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36476 (request call id 40) took 1084 ms. Trace:
I20260809 07:32:27.380112 31038 rpcz_store.cc:276] 0809 07:32:26.263666 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.893012 (+629346us) service_pool.cc:227] Handling call
0809 07:32:26.902198 (+ 9186us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.902308 (+ 110us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.057780 (+155472us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.274026 (+216246us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.347900 (+ 73874us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b5800004598. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99e82 00007f409aeb28b9 00007f4099dc55d8 00007f4099e89063 00007f4099e84e2f 00007f4099e855ed 00007f4099e36869 00007f4093438c44 00007f4093438bd2
0809 07:32:27.348179 (+ 279us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":67280256}
I20260809 07:32:27.391100 31029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a18497352404d2491d49a9241fe61bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.382280 31038 tablet_service.cc:1467] Processing CreateTablet for tablet 31954ed4d71d4a6796aa3c4bb2ded761 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260809 07:32:27.400684 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31954ed4d71d4a6796aa3c4bb2ded761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.403388 31034 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba1c2ba04d7459b93ff369f6223cef6 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260809 07:32:27.394582 31244 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.388676 31245 tablet_bootstrap.cc:492] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.407600 31245 ts_tablet_manager.cc:1397] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.280s user 0.007s sys 0.016s
I20260809 07:32:27.408519 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:27.412257 30880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e87f17b17dd4e1fba73ed247d6b4039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.373477 30885 rpcz_store.cc:276] 0809 07:32:26.229056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:27.057919 (+828863us) service_pool.cc:227] Handling call
0809 07:32:27.266133 (+208214us) spinlock_profiling.cc:243] Waited 964 us on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099ea7d21 00007f4099e851e3 00007f4099e36869 00007f4093438c44 00007f4093438bd2 00007f4093438b01
0809 07:32:27.266268 (+ 135us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:27.266385 (+ 117us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.334666 (+ 68281us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.371029 (+ 36363us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.371420 (+ 391us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2217216}
W20260809 07:32:27.414978 31120 leader_election.cc:341] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.420020 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 51e40c75108c4a86a26730860edee159 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260809 07:32:27.420962 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f1326c9441f41f4ad3abaac9ca2b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.429538 30875 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 40) took 1186 ms. Trace:
I20260809 07:32:27.404685 31034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba1c2ba04d7459b93ff369f6223cef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.434662 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51e40c75108c4a86a26730860edee159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.435300 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 59bd1fdaac4346aa9f2f9be0f904b408 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260809 07:32:27.446113 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59bd1fdaac4346aa9f2f9be0f904b408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.429958 30875 rpcz_store.cc:276] 0809 07:32:26.242681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:27.086140 (+843459us) service_pool.cc:227] Handling call
0809 07:32:27.123019 (+ 36879us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:27.127229 (+ 4210us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.318717 (+191488us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.429016 (+110299us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.429472 (+ 456us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.430595 31244 ts_tablet_manager.cc:1428] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.098s user 0.007s sys 0.000s
I20260809 07:32:27.431659 31245 raft_consensus.cc:348] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.451349 31245 raft_consensus.cc:374] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.451660 31245 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.452358 31245 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.456208 31244 tablet_bootstrap.cc:492] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.456964 31245 ts_tablet_manager.cc:1428] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.049s user 0.003s sys 0.003s
I20260809 07:32:27.457908 31245 tablet_bootstrap.cc:492] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:27.462203 31244 tablet_bootstrap.cc:654] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.480859 30889 tablet_service.cc:1467] Processing CreateTablet for tablet 723b3b7ed7b749d3b4b448d22577a8e7 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260809 07:32:27.489502 31243 tablet_bootstrap.cc:654] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.490070 30889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 723b3b7ed7b749d3b4b448d22577a8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.436390 30883 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 41) took 1171 ms. Trace:
I20260809 07:32:27.490636 30883 rpcz_store.cc:276] 0809 07:32:26.265106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:27.192812 (+927706us) service_pool.cc:227] Handling call
0809 07:32:27.195879 (+ 3067us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:27.195988 (+ 109us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.354526 (+158538us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.435929 (+ 81403us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.436337 (+ 408us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.492519 30883 tablet_service.cc:1467] Processing CreateTablet for tablet 3bbfdffc27ac46a0a18582f85cd4a00a (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260809 07:32:27.439877 31033 tablet_service.cc:1467] Processing CreateTablet for tablet cb8db10c4bb04e86a15bcbff5ffed3d5 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260809 07:32:27.480859 30875 tablet_service.cc:1467] Processing CreateTablet for tablet b72ff16aedce4676b889395ca4d8909d (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260809 07:32:27.498342 31033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8db10c4bb04e86a15bcbff5ffed3d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.511022 31252 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.511554 31252 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.519570 31252 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:27.522833 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:27.523491 30911 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:27.529806 30882 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 39) took 1300 ms. Trace:
W20260809 07:32:27.526647 30724 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 47799.9us
I20260809 07:32:27.521430 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbfdffc27ac46a0a18582f85cd4a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.530179 30882 rpcz_store.cc:276] 0809 07:32:26.229709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:27.077000 (+847291us) service_pool.cc:227] Handling call
0809 07:32:27.092163 (+ 15163us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:27.092253 (+ 90us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.339206 (+246953us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.529312 (+190106us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.529750 (+ 438us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.542848 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72ff16aedce4676b889395ca4d8909d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.549587 31244 tablet_bootstrap.cc:492] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.562896 31244 ts_tablet_manager.cc:1397] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.107s user 0.009s sys 0.007s
I20260809 07:32:27.543318 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:27.563892 31059 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:27.553344 31159 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45792 (request call id 55) took 1019 ms. Trace:
I20260809 07:32:27.564962 31244 raft_consensus.cc:348] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.556347 30881 tablet_service.cc:1467] Processing CreateTablet for tablet 31954ed4d71d4a6796aa3c4bb2ded761 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260809 07:32:27.543892 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:27.566426 31244 raft_consensus.cc:374] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.540449 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:27.566645 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31954ed4d71d4a6796aa3c4bb2ded761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.566745 31244 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.567438 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.567627 31244 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.568473 31252 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:27.568804 31252 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:27.569089 31252 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.543912 30882 tablet_service.cc:1467] Processing CreateTablet for tablet c2c9e03f3fb24c0eba0bf8f658070202 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260809 07:32:27.571635 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c9e03f3fb24c0eba0bf8f658070202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.559206 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba1c2ba04d7459b93ff369f6223cef6 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260809 07:32:27.564834 31159 rpcz_store.cc:276] 0809 07:32:26.534045 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:27.217892 (+683847us) service_pool.cc:227] Handling call
0809 07:32:27.336326 (+118434us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:27.336428 (+ 102us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.409410 (+ 72982us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.552820 (+143410us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.553282 (+ 462us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.560366 30880 tablet_service.cc:1467] Processing CreateTablet for tablet cb8db10c4bb04e86a15bcbff5ffed3d5 (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260809 07:32:27.576738 31244 ts_tablet_manager.cc:1428] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20260809 07:32:27.556914 31245 tablet_bootstrap.cc:654] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.574898 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba1c2ba04d7459b93ff369f6223cef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.538265 30877 tablet_service.cc:1467] Processing CreateTablet for tablet 1a18497352404d2491d49a9241fe61bf (DEFAULT_TABLE table=test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260809 07:32:27.581557 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a18497352404d2491d49a9241fe61bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.583870 31244 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.587174 30880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb8db10c4bb04e86a15bcbff5ffed3d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:32:27.589596 31244 tablet_bootstrap.cc:654] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:27.591301 31120 leader_election.cc:341] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.593864 31252 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.596628 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:27.597234 30911 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.591217 31288 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:27.600241 31288 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:27.600572 31288 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.603736 31244 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.604314 31244 ts_tablet_manager.cc:1397] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.011s
I20260809 07:32:27.603719 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:27.606431 31120 leader_election.cc:341] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.606880 31244 raft_consensus.cc:348] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.607568 31244 raft_consensus.cc:374] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.607808 31243 tablet_bootstrap.cc:492] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.607939 31244 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.607607 31252 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:27.608348 31243 ts_tablet_manager.cc:1397] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.236s user 0.008s sys 0.011s
I20260809 07:32:27.614711 31245 tablet_bootstrap.cc:492] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.615171 31245 ts_tablet_manager.cc:1397] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.157s user 0.014s sys 0.000s
I20260809 07:32:27.616966 31243 raft_consensus.cc:348] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.617683 31243 raft_consensus.cc:374] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.618017 31243 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.617631 31245 raft_consensus.cc:348] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.618850 31243 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.618870 31245 raft_consensus.cc:374] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.620137 31245 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.620952 30911 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:27.624131 31288 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.608785 31244 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.627744 30885 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43208 (request call id 57) took 1002 ms. Trace:
I20260809 07:32:27.628089 31253 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.628154 30885 rpcz_store.cc:276] 0809 07:32:26.625316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:27.558338 (+933022us) service_pool.cc:227] Handling call
0809 07:32:27.574463 (+ 16125us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:27.574566 (+ 103us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.615538 (+ 40972us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.627280 (+ 11742us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.627689 (+ 409us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:27.629069 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:27.630055 31244 ts_tablet_manager.cc:1428] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.025s user 0.004s sys 0.002s
I20260809 07:32:27.631012 31244 tablet_bootstrap.cc:492] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.631559 31252 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:27.630198 31288 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:27.628628 31253 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.634348 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:27.634835 31253 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.634943 31058 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.636376 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.636679 31244 tablet_bootstrap.cc:654] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:27.637787 30853 leader_election.cc:341] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.638964 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:27.639483 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:27.640316 31120 leader_election.cc:341] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:27.642014 30852 leader_election.cc:341] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.642553 30852 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:27.643379 31253 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.646935 31244 tablet_bootstrap.cc:492] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.647436 31244 ts_tablet_manager.cc:1397] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260809 07:32:27.650147 31244 raft_consensus.cc:348] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.651072 31244 raft_consensus.cc:374] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.651352 31244 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.651859 31254 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.652319 31254 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.653107 31243 ts_tablet_manager.cc:1428] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260809 07:32:27.652628 31244 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.654050 31243 tablet_bootstrap.cc:492] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:27.620870 31245 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.656620 31245 ts_tablet_manager.cc:1428] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260809 07:32:27.657397 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:27.659498 31006 leader_election.cc:341] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.654215 31254 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.660100 31243 tablet_bootstrap.cc:654] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.663245 31254 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.663796 31254 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.665303 31252 raft_consensus.cc:686] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:27.666194 31252 consensus_queue.cc:237] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.668045 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:27.669472 31006 leader_election.cc:341] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.670058 31254 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.651335 31058 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:27.671447 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.672845 31294 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:27.673763 31294 raft_consensus.cc:686] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:27.674692 31294 consensus_queue.cc:237] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.675472 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:27.676905 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:27.678439 31005 leader_election.cc:341] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.679162 31005 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:27.680497 31005 leader_election.cc:341] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.680891 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.684329 31243 tablet_bootstrap.cc:492] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.684702 31243 ts_tablet_manager.cc:1397] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.031s user 0.017s sys 0.003s
I20260809 07:32:27.681774 31254 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.685737 31254 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.687362 31244 ts_tablet_manager.cc:1428] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.040s user 0.005s sys 0.023s
I20260809 07:32:27.687150 31243 raft_consensus.cc:348] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.687872 31243 raft_consensus.cc:374] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.688153 31243 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.689042 31245 tablet_bootstrap.cc:492] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:27.689304 30745 catalog_manager.cc:5495] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:27.690840 31244 tablet_bootstrap.cc:492] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.697805 31245 tablet_bootstrap.cc:654] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.697882 31244 tablet_bootstrap.cc:654] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.700932 31243 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.705797 31244 tablet_bootstrap.cc:492] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.706388 31244 ts_tablet_manager.cc:1397] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
I20260809 07:32:27.707199 31243 ts_tablet_manager.cc:1428] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260809 07:32:27.708220 31243 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:27.709429 31244 raft_consensus.cc:348] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.710309 31244 raft_consensus.cc:374] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.710692 31244 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.711470 31244 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.717175 31245 tablet_bootstrap.cc:492] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.717634 31245 ts_tablet_manager.cc:1397] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.005s
I20260809 07:32:27.718430 31244 ts_tablet_manager.cc:1428] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.012s user 0.005s sys 0.002s
I20260809 07:32:27.719492 31244 tablet_bootstrap.cc:492] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.720254 31245 raft_consensus.cc:348] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.720976 31245 raft_consensus.cc:374] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.721257 31245 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.722227 31245 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.726014 31244 tablet_bootstrap.cc:654] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.726899 31243 tablet_bootstrap.cc:654] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.727875 31245 ts_tablet_manager.cc:1428] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.010s user 0.002s sys 0.005s
I20260809 07:32:27.728811 31245 tablet_bootstrap.cc:492] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:27.735774 31245 tablet_bootstrap.cc:654] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.752040 31243 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.752496 31243 ts_tablet_manager.cc:1397] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.044s user 0.011s sys 0.009s
I20260809 07:32:27.757956 31244 tablet_bootstrap.cc:492] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.758460 31244 ts_tablet_manager.cc:1397] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.039s user 0.017s sys 0.020s
I20260809 07:32:27.758596 31245 tablet_bootstrap.cc:492] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.759181 31245 ts_tablet_manager.cc:1397] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.000s
I20260809 07:32:27.758394 30745 catalog_manager.cc:5495] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:27.761106 31244 raft_consensus.cc:348] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.761840 31244 raft_consensus.cc:374] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.762120 31244 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.762897 31244 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.754930 31243 raft_consensus.cc:348] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.763916 31243 raft_consensus.cc:374] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.764253 31243 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.765304 31244 ts_tablet_manager.cc:1428] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:32:27.763430 31245 raft_consensus.cc:348] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.766225 31245 raft_consensus.cc:374] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.766539 31245 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.764978 31243 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.768846 31243 ts_tablet_manager.cc:1428] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260809 07:32:27.769657 31245 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.772200 31245 ts_tablet_manager.cc:1428] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.013s user 0.004s sys 0.001s
I20260809 07:32:27.772855 31243 tablet_bootstrap.cc:492] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:27.772909 31297 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.773444 31297 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.775266 31244 tablet_bootstrap.cc:492] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.775476 31297 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.777163 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.777779 31181 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:27.779013 31006 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:27.779361 31243 tablet_bootstrap.cc:654] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.780885 31297 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:27.781253 31297 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:27.781664 31297 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.788719 31297 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.781366 31244 tablet_bootstrap.cc:654] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.791276 31243 tablet_bootstrap.cc:492] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.791743 31243 ts_tablet_manager.cc:1397] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.009s
I20260809 07:32:27.791924 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:27.792595 31181 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.795477 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:27.796224 30909 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:27.796965 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:27.797518 30911 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.799764 31254 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.799469 31243 raft_consensus.cc:348] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.800213 31254 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.802182 31254 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.800213 31243 raft_consensus.cc:374] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.802909 31243 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.803320 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.803927 31182 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:27.803567 31243 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.806015 31297 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.806160 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:27.808485 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:27.808763 31245 tablet_bootstrap.cc:492] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
W20260809 07:32:27.809669 31005 leader_election.cc:341] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.811317 30911 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:27.808039 31181 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:27.812994 31243 ts_tablet_manager.cc:1428] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.021s user 0.000s sys 0.006s
I20260809 07:32:27.812881 31006 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:27.813678 31254 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:27.809965 31297 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:27.819708 31244 tablet_bootstrap.cc:492] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.819661 31243 tablet_bootstrap.cc:492] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:27.820324 31244 ts_tablet_manager.cc:1397] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.045s user 0.017s sys 0.006s
I20260809 07:32:27.820772 31245 tablet_bootstrap.cc:654] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.826140 31244 raft_consensus.cc:348] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.826997 31244 raft_consensus.cc:374] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.827308 31244 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.829072 31243 tablet_bootstrap.cc:654] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.828003 31244 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.818969 31297 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:27.834798 31254 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:27.840188 31254 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.841670 31327 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.842104 31327 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.844017 31327 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.836488 31294 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.847250 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:27.850184 31005 leader_election.cc:341] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.852381 31294 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.852831 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.853633 31181 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:27.854395 31294 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:32:27.854863 31006 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.837275 31297 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.858850 31243 tablet_bootstrap.cc:492] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.859293 31243 ts_tablet_manager.cc:1397] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.041s user 0.011s sys 0.002s
I20260809 07:32:27.861922 31243 raft_consensus.cc:348] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.862627 31243 raft_consensus.cc:374] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.862947 31243 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.862775 31297 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.863898 31243 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.865871 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:27.867769 31120 leader_election.cc:341] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.868678 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:27.870579 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:27.870911 31005 leader_election.cc:341] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.869679 31254 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:27.871556 31058 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:27.871850 31254 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:27.872251 31254 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.873914 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:27.874704 31181 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.874771 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.876936 31294 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:27.877508 31294 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:27.878046 31294 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.879920 31245 tablet_bootstrap.cc:492] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.880385 31245 ts_tablet_manager.cc:1397] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.072s user 0.006s sys 0.007s
I20260809 07:32:27.882045 30742 catalog_manager.cc:5495] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:27.886222 31244 ts_tablet_manager.cc:1428] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.066s user 0.010s sys 0.010s
I20260809 07:32:27.887207 31244 tablet_bootstrap.cc:492] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.882934 31245 raft_consensus.cc:348] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.888392 31245 raft_consensus.cc:374] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.888650 31245 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.889377 31245 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.892624 31244 tablet_bootstrap.cc:654] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.898636 31245 ts_tablet_manager.cc:1428] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260809 07:32:27.899611 31245 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:27.902999 31294 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.910491 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:27.911044 31058 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.912907 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:27.914383 31120 leader_election.cc:341] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.914979 31294 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:32:27.905224 31245 tablet_bootstrap.cc:654] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.906034 31243 ts_tablet_manager.cc:1428] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.046s user 0.004s sys 0.003s
I20260809 07:32:27.917440 31243 tablet_bootstrap.cc:492] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:27.906435 31181 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:27.907060 31254 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.920931 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.921515 31254 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.922055 31254 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:27.922521 31254 raft_consensus.cc:686] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:27.922863 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:27.923398 31243 tablet_bootstrap.cc:654] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.923455 31182 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.924567 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:27.926084 31005 leader_election.cc:341] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:27.923354 31254 consensus_queue.cc:237] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.931519 31244 tablet_bootstrap.cc:492] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:27.908123 31297 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.931986 31244 ts_tablet_manager.cc:1397] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.009s
I20260809 07:32:27.931546 31182 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:27.933579 31006 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.934755 31297 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:27.935657 31297 raft_consensus.cc:686] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:27.936293 31253 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.936739 31253 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.934485 31244 raft_consensus.cc:348] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.940073 31244 raft_consensus.cc:374] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.940341 31244 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.942024 31244 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.942756 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:27.940492 31253 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.938422 31058 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:27.945425 31244 ts_tablet_manager.cc:1428] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260809 07:32:27.946045 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.948246 31182 raft_consensus.cc:2379] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:27.947181 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:27.946421 31244 tablet_bootstrap.cc:492] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:27.949643 31294 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:27.950098 31294 raft_consensus.cc:686] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:27.936547 31297 consensus_queue.cc:237] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.950932 30853 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:27.950927 31294 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.952250 31253 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.959565 31253 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.960244 31253 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.960733 31253 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.963490 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:27.964329 31058 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:27.966939 31253 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:27.967691 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:27.969389 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:27.983213 31245 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:27.983673 31245 ts_tablet_manager.cc:1397] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.084s user 0.009s sys 0.019s
I20260809 07:32:27.985420 31244 tablet_bootstrap.cc:654] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:27.984720 30742 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:27.986100 31245 raft_consensus.cc:348] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.986775 31245 raft_consensus.cc:374] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.987030 31245 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.987713 31245 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.974885 31253 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:27.992911 31243 tablet_bootstrap.cc:492] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:27.993384 31243 ts_tablet_manager.cc:1397] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.076s user 0.030s sys 0.009s
I20260809 07:32:27.994858 30745 catalog_manager.cc:5495] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:27.995915 31253 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:27.996615 31253 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:27.996464 31243 raft_consensus.cc:348] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.997278 31243 raft_consensus.cc:374] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:27.997042 31253 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:27.997565 31243 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:27.998342 31243 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.000104 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.000247 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.000754 31058 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
W20260809 07:32:28.001602 30853 leader_election.cc:341] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.002198 31253 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.002429 30852 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.004460 31245 ts_tablet_manager.cc:1428] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260809 07:32:28.005127 31253 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.005503 31253 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.005806 31253 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.010648 31244 tablet_bootstrap.cc:492] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.011130 31244 ts_tablet_manager.cc:1397] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.002s
I20260809 07:32:28.011597 31245 tablet_bootstrap.cc:492] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.013072 31253 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.013424 31244 raft_consensus.cc:348] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.014055 31244 raft_consensus.cc:374] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.014430 31244 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.000329 31243 ts_tablet_manager.cc:1428] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 07:32:28.015030 31253 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.015039 31244 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.015829 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.016390 31058 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.017201 31244 ts_tablet_manager.cc:1428] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:32:28.012530 30744 catalog_manager.cc:5495] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:28.018066 31244 tablet_bootstrap.cc:492] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.019863 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:28.021193 30853 leader_election.cc:341] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.022346 31243 tablet_bootstrap.cc:492] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.028918 31243 tablet_bootstrap.cc:654] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.026516 31058 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:28.035755 31245 tablet_bootstrap.cc:654] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.035516 30852 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.028934 31244 tablet_bootstrap.cc:654] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.041388 31253 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.042920 31345 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.043378 31345 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.045575 31345 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.046730 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.047009 31244 tablet_bootstrap.cc:492] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.047086 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.047595 31244 ts_tablet_manager.cc:1397] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.000s
I20260809 07:32:28.047873 31058 raft_consensus.cc:2379] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:28.047636 31253 raft_consensus.cc:686] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:28.049057 30852 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.050246 31345 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.050707 31245 tablet_bootstrap.cc:492] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.050426 31244 raft_consensus.cc:348] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.051120 31245 ts_tablet_manager.cc:1397] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.040s user 0.013s sys 0.002s
I20260809 07:32:28.053247 31253 consensus_queue.cc:237] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.051120 31244 raft_consensus.cc:374] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.056254 31244 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.056951 31244 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.053876 31245 raft_consensus.cc:348] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.057693 31245 raft_consensus.cc:374] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.057958 31245 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.058537 31294 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.058789 31245 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.059180 31294 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.061973 31244 ts_tablet_manager.cc:1428] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260809 07:32:28.062907 31244 tablet_bootstrap.cc:492] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.064680 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.065392 30911 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.066885 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:28.067811 31252 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.068154 31252 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.068490 31252 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.069422 31244 tablet_bootstrap.cc:654] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.073637 31243 tablet_bootstrap.cc:492] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.074044 31243 ts_tablet_manager.cc:1397] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.052s user 0.036s sys 0.007s
I20260809 07:32:28.076395 31243 raft_consensus.cc:348] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.077060 31243 raft_consensus.cc:374] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.077329 31243 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.077634 31288 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.078081 31243 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.082015 31254 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.082512 31254 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.084638 31254 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.086229 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.087570 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.088172 30909 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:28.089313 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.090364 31254 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.090694 31254 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.090998 31254 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.093576 31252 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.096560 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.098030 31182 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:28.099596 31294 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.101956 31254 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.104092 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.104640 30909 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.104995 31058 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:28.111433 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.112296 31058 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:28.114238 31252 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.115141 31345 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.124627 30909 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.125180 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.126051 30906 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:28.125924 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.124676 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:28.127381 31182 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:28.129130 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:28.130021 31297 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.130776 31252 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.131281 31252 raft_consensus.cc:686] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:28.132051 31252 consensus_queue.cc:237] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.135807 31254 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.136618 31244 tablet_bootstrap.cc:492] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.136823 31245 ts_tablet_manager.cc:1428] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.085s user 0.019s sys 0.023s
I20260809 07:32:28.137051 31244 ts_tablet_manager.cc:1397] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.074s user 0.007s sys 0.004s
I20260809 07:32:28.137657 31245 tablet_bootstrap.cc:492] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.138958 31243 ts_tablet_manager.cc:1428] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.065s user 0.014s sys 0.004s
I20260809 07:32:28.139874 31243 tablet_bootstrap.cc:492] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.141649 31244 raft_consensus.cc:348] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.144750 31245 tablet_bootstrap.cc:654] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.145293 31244 raft_consensus.cc:374] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.145175 31294 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.145609 31244 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.146981 31363 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:28.146570 31244 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.149896 31244 ts_tablet_manager.cc:1428] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260809 07:32:28.150601 31244 tablet_bootstrap.cc:492] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.151258 31297 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.151801 31297 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.153967 31297 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.155467 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:28.157753 31006 leader_election.cc:341] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.157939 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:28.170461 31005 leader_election.cc:341] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.171311 31005 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:28.171751 31245 tablet_bootstrap.cc:492] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.174328 31297 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.175933 31245 ts_tablet_manager.cc:1397] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.038s user 0.012s sys 0.005s
I20260809 07:32:28.176985 31354 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.177500 31354 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.179189 31244 tablet_bootstrap.cc:654] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.179735 31245 raft_consensus.cc:348] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.180408 31245 raft_consensus.cc:374] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.180220 31354 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.180668 31245 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.181283 31245 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.182087 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.184165 31245 ts_tablet_manager.cc:1428] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 07:32:28.185559 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:28.186964 30852 leader_election.cc:341] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.187569 31243 tablet_bootstrap.cc:654] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.189392 31358 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:28.193389 30853 leader_election.cc:341] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.194645 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.198565 30744 catalog_manager.cc:5495] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:28.201016 31245 tablet_bootstrap.cc:492] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.212580 31252 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.212844 31059 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.215263 31252 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.217790 31252 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.199542 31354 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.193522 31358 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.220822 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.222522 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.223547 31057 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.225539 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.230801 31363 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.231361 31363 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.231843 31363 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.181777 30741 catalog_manager.cc:5495] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:28.255487 31254 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:28.258476 31252 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:28.251928 30909 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.274133 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.274255 31358 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.274832 31057 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:28.276127 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.277769 30852 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.278997 31358 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.279353 31358 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.279663 31358 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.281762 31363 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.282941 31182 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:28.283776 31363 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.287437 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.288355 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.288900 31057 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.289685 31245 tablet_bootstrap.cc:654] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.292589 31254 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.300580 31244 tablet_bootstrap.cc:492] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.315536 31244 ts_tablet_manager.cc:1397] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.165s user 0.040s sys 0.033s
I20260809 07:32:28.300851 31057 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.319427 31244 raft_consensus.cc:348] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.320122 31244 raft_consensus.cc:374] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.320397 31244 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.321000 31244 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.324195 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.325179 31294 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.325588 31244 ts_tablet_manager.cc:1428] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260809 07:32:28.328740 31244 tablet_bootstrap.cc:492] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.328799 31294 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:28.336833 31294 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.346338 31358 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.347381 31243 tablet_bootstrap.cc:492] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.348280 30909 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:28.349319 31243 ts_tablet_manager.cc:1397] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.210s user 0.013s sys 0.007s
W20260809 07:32:28.350744 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:28.362658 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.363777 31059 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:28.365569 31244 tablet_bootstrap.cc:654] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.370036 31358 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.372656 31245 tablet_bootstrap.cc:492] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.373045 31245 ts_tablet_manager.cc:1397] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.172s user 0.010s sys 0.005s
I20260809 07:32:28.373551 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:28.374426 31223 tablet.cc:2369] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:32:28.375015 30853 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.376327 31358 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.377056 31243 raft_consensus.cc:348] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.377745 31243 raft_consensus.cc:374] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.378031 31243 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.378787 31243 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.380846 31243 ts_tablet_manager.cc:1428] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.031s user 0.004s sys 0.000s
I20260809 07:32:28.381757 31243 tablet_bootstrap.cc:492] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.385037 31332 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 07:32:28.394835 30742 catalog_manager.cc:5495] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:28.395547 31245 raft_consensus.cc:348] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.397873 31245 raft_consensus.cc:374] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.398263 31245 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.399137 31245 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.401144 31245 ts_tablet_manager.cc:1428] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.028s user 0.002s sys 0.003s
I20260809 07:32:28.404018 31326 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 07:32:28.405406 31244 tablet_bootstrap.cc:492] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.405928 31244 ts_tablet_manager.cc:1397] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.078s user 0.009s sys 0.032s
I20260809 07:32:28.406971 31245 tablet_bootstrap.cc:492] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.409502 31244 raft_consensus.cc:348] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.410022 31244 raft_consensus.cc:374] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.410240 31244 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.412925 31244 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.414860 31244 ts_tablet_manager.cc:1428] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 07:32:28.415259 31294 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.415659 31244 tablet_bootstrap.cc:492] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.415710 31294 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.417769 31294 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.418833 31243 tablet_bootstrap.cc:654] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.419723 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.422689 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.424445 31059 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.423328 30906 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.429117 31245 tablet_bootstrap.cc:654] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.429540 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.431780 31252 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.432116 31252 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.432423 31252 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.433759 31294 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.442165 31252 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.444169 31252 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.447415 31244 tablet_bootstrap.cc:654] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:28.450781 31005 consensus_peers.cc:487] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a -> Peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Couldn't send request to peer 2c9de56c3fa3487c9dac990675c34be5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:28.453404 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.453999 31059 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.453877 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.456799 30906 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.458912 31358 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.461798 31059 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.462952 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.463299 31243 tablet_bootstrap.cc:492] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.465492 31252 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.465950 31252 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:28.466431 31243 ts_tablet_manager.cc:1397] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.085s user 0.008s sys 0.007s
I20260809 07:32:28.466775 31252 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.468814 31243 raft_consensus.cc:348] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.469497 31243 raft_consensus.cc:374] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.469745 31243 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.470361 31243 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.472229 31243 ts_tablet_manager.cc:1428] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260809 07:32:28.473044 31243 tablet_bootstrap.cc:492] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.473516 31245 tablet_bootstrap.cc:492] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.474080 31245 ts_tablet_manager.cc:1397] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.005s
I20260809 07:32:28.478579 31181 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.482340 30906 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.482745 31244 tablet_bootstrap.cc:492] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.483187 31244 ts_tablet_manager.cc:1397] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.002s
I20260809 07:32:28.485402 31244 raft_consensus.cc:348] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.486011 31244 raft_consensus.cc:374] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.486354 31244 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.486233 31288 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:28.487116 31244 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.490648 31294 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.490825 31253 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.491145 31294 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.492695 31244 ts_tablet_manager.cc:1428] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260809 07:32:28.493530 31244 tablet_bootstrap.cc:492] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.496068 31294 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.498416 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.500718 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.502164 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.503863 31244 tablet_bootstrap.cc:654] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.508157 31243 tablet_bootstrap.cc:654] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.511152 31245 raft_consensus.cc:348] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.513047 31326 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.513501 31326 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.511828 31245 raft_consensus.cc:374] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.514878 31245 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.515190 31326 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.517469 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.518473 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:28.519681 31006 leader_election.cc:341] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.524597 31294 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:28.525898 31245 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.528056 31245 ts_tablet_manager.cc:1428] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.053s user 0.004s sys 0.000s
I20260809 07:32:28.528911 31245 tablet_bootstrap.cc:492] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.529403 31326 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.531385 30909 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:28.534078 31245 tablet_bootstrap.cc:654] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.545745 31253 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.546263 31253 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.548228 31253 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.548113 30742 catalog_manager.cc:5495] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:28.551518 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:28.554306 30853 leader_election.cc:341] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.557202 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.558429 31326 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.558776 31326 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.559070 31326 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.560110 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.560799 31058 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:28.563452 30852 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.565531 31253 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.567736 31253 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.568181 31253 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.570006 31326 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.572674 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.573009 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:28.573289 30909 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:28.574415 31006 leader_election.cc:341] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.577287 31244 tablet_bootstrap.cc:492] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.577723 31244 ts_tablet_manager.cc:1397] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.084s user 0.019s sys 0.008s
I20260809 07:32:28.578405 31326 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.580217 31244 raft_consensus.cc:348] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.580418 30909 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:28.580873 31244 raft_consensus.cc:374] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.581152 31244 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.581676 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.581739 31244 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.584417 31243 tablet_bootstrap.cc:492] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.584847 31243 ts_tablet_manager.cc:1397] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.112s user 0.008s sys 0.019s
I20260809 07:32:28.587118 31243 raft_consensus.cc:348] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.587754 31243 raft_consensus.cc:374] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.588013 31243 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.588562 31243 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.590736 31243 ts_tablet_manager.cc:1428] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260809 07:32:28.590665 31253 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.591619 31243 tablet_bootstrap.cc:492] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.592715 31245 tablet_bootstrap.cc:492] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.593125 31245 ts_tablet_manager.cc:1397] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.002s
I20260809 07:32:28.593502 31253 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.595340 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:28.595752 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.596331 31058 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:28.596724 30853 leader_election.cc:341] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.597154 31332 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.597602 31332 raft_consensus.cc:686] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:28.598431 31332 consensus_queue.cc:237] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.605413 31058 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:28.595865 31245 raft_consensus.cc:348] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.606165 31245 raft_consensus.cc:374] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.606467 31245 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.606669 30852 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.607220 31244 ts_tablet_manager.cc:1428] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.029s user 0.004s sys 0.003s
I20260809 07:32:28.607790 31253 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.608084 31244 tablet_bootstrap.cc:492] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.609081 31253 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:28.609972 31253 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.610263 31245 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.622392 31245 ts_tablet_manager.cc:1428] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.029s user 0.006s sys 0.003s
I20260809 07:32:28.623309 31245 tablet_bootstrap.cc:492] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.626852 31243 tablet_bootstrap.cc:654] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.628523 31245 tablet_bootstrap.cc:654] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.630525 31244 tablet_bootstrap.cc:654] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.630745 30744 catalog_manager.cc:5495] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:28.639518 31244 tablet_bootstrap.cc:492] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.639990 31244 ts_tablet_manager.cc:1397] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.007s
I20260809 07:32:28.645581 31243 tablet_bootstrap.cc:492] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.646055 31243 ts_tablet_manager.cc:1397] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.003s
I20260809 07:32:28.648495 31243 raft_consensus.cc:348] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.649187 31243 raft_consensus.cc:374] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.649437 31243 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.650228 31243 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.651127 31244 raft_consensus.cc:348] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.652454 30741 catalog_manager.cc:5495] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:28.656636 31294 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.651803 31244 raft_consensus.cc:374] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.664714 31244 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.665374 31244 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.666782 31243 ts_tablet_manager.cc:1428] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260809 07:32:28.667225 31244 ts_tablet_manager.cc:1428] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260809 07:32:28.667760 31243 tablet_bootstrap.cc:492] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.668020 31244 tablet_bootstrap.cc:492] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.674153 31244 tablet_bootstrap.cc:654] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.678045 31243 tablet_bootstrap.cc:654] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.683519 31363 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.686782 31245 tablet_bootstrap.cc:492] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.687196 31245 ts_tablet_manager.cc:1397] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.064s user 0.026s sys 0.021s
I20260809 07:32:28.689455 31245 raft_consensus.cc:348] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.690081 31245 raft_consensus.cc:374] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.690326 31245 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.691174 31245 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.693142 31245 ts_tablet_manager.cc:1428] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:32:28.693979 31245 tablet_bootstrap.cc:492] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.699282 31245 tablet_bootstrap.cc:654] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.703976 31243 tablet_bootstrap.cc:492] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.704438 31243 ts_tablet_manager.cc:1397] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.007s
I20260809 07:32:28.710868 31243 raft_consensus.cc:348] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.712220 31243 raft_consensus.cc:374] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.712463 31243 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.713320 31243 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.716619 31243 ts_tablet_manager.cc:1428] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260809 07:32:28.723326 31245 tablet_bootstrap.cc:492] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.725157 31245 ts_tablet_manager.cc:1397] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.007s
I20260809 07:32:28.727694 31245 raft_consensus.cc:348] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.728482 31245 raft_consensus.cc:374] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.728783 31245 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.729504 31245 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.732957 31245 ts_tablet_manager.cc:1428] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:32:28.733901 31245 tablet_bootstrap.cc:492] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.733830 31294 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.734228 31243 tablet_bootstrap.cc:492] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.734457 31294 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.736388 31294 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.738628 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.739248 30906 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.739804 31243 tablet_bootstrap.cc:654] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.740180 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.742586 31058 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.742831 31245 tablet_bootstrap.cc:654] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.741941 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:28.749518 31294 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.749887 31294 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.750202 31294 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.754657 31288 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.756166 31243 tablet_bootstrap.cc:492] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.756609 31243 ts_tablet_manager.cc:1397] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.004s
I20260809 07:32:28.759073 31243 raft_consensus.cc:348] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.759727 31243 raft_consensus.cc:374] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.759974 31243 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.760721 31243 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.764047 31245 tablet_bootstrap.cc:492] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.764549 31245 ts_tablet_manager.cc:1397] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.013s
I20260809 07:32:28.764832 31294 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.766989 31245 raft_consensus.cc:348] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.767695 31245 raft_consensus.cc:374] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.767990 31245 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.767187 31243 ts_tablet_manager.cc:1428] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260809 07:32:28.768708 31245 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.769579 31243 tablet_bootstrap.cc:492] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.771234 31245 ts_tablet_manager.cc:1428] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:32:28.772188 31245 tablet_bootstrap.cc:492] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.777130 31294 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.784533 31244 tablet_bootstrap.cc:492] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.785002 31244 ts_tablet_manager.cc:1397] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.117s user 0.020s sys 0.014s
I20260809 07:32:28.789274 31245 tablet_bootstrap.cc:654] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.793848 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.797845 31058 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.794880 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.797377 31243 tablet_bootstrap.cc:654] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.802476 31363 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:28.802767 30909 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.809562 30909 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.812023 31332 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.812815 31332 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.814987 31332 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.816123 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.816437 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.816936 30906 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
W20260809 07:32:28.817564 31006 leader_election.cc:341] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.818756 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.819965 31332 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.819857 31244 raft_consensus.cc:348] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.820436 31332 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.821017 31332 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.824934 31363 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.811990 31058 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.825417 31363 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.827435 31363 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.820545 31244 raft_consensus.cc:374] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.828058 31244 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.830300 31288 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.834609 31288 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.836619 31288 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.830835 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.830914 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:28.837785 31252 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.838281 31252 raft_consensus.cc:686] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:28.839100 31252 consensus_queue.cc:237] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.844864 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.845748 30906 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:28.847518 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:28.848562 31058 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:28.849874 31244 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.850453 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.831842 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:28.851832 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:28.852386 31363 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.854408 31244 ts_tablet_manager.cc:1428] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.069s user 0.007s sys 0.000s
I20260809 07:32:28.856151 31253 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.856621 31253 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.857695 31294 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.859486 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:28.857823 31120 leader_election.cc:341] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.860266 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.858740 31253 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:28.860884 30852 leader_election.cc:341] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.860888 31181 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:28.858146 31294 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.862290 31294 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.862511 31363 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.863575 30853 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.864915 31253 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:28.865285 31253 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:28.865612 31253 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.872742 31253 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.875339 31294 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.877421 31294 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:28.878446 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:28.879069 31057 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.881040 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.834095 31332 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:28.882386 31120 leader_election.cc:341] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.882593 31297 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.883080 31297 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.884282 31244 tablet_bootstrap.cc:492] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.884984 31297 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.887290 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:28.888684 31005 leader_election.cc:341] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.889256 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:28.890431 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:28.891506 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:28.891834 31006 leader_election.cc:341] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.892274 31006 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:28.892815 30852 leader_election.cc:341] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:28.893324 31006 leader_election.cc:341] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.893975 31297 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.894203 31253 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.892767 31057 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:28.895859 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:28.896955 31252 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.897444 31252 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:28.897588 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:28.898190 31181 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.898371 31252 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.905195 31181 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:28.905717 31244 tablet_bootstrap.cc:654] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.906571 30853 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:28.907321 31243 tablet_bootstrap.cc:492] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.910738 31243 ts_tablet_manager.cc:1397] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.141s user 0.037s sys 0.027s
I20260809 07:32:28.911676 31345 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.912132 31345 raft_consensus.cc:686] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:28.913038 31243 raft_consensus.cc:348] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.913682 31243 raft_consensus.cc:374] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.913925 31243 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.914482 31243 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.912899 31345 consensus_queue.cc:237] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.916903 31243 ts_tablet_manager.cc:1428] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:32:28.917744 31243 tablet_bootstrap.cc:492] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.922679 31243 tablet_bootstrap.cc:654] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.928704 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:28.929353 30909 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:28.934216 31244 tablet_bootstrap.cc:492] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:28.883410 31332 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.934751 31244 ts_tablet_manager.cc:1397] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.007s
I20260809 07:32:28.937913 31243 tablet_bootstrap.cc:492] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:28.938354 31243 ts_tablet_manager.cc:1397] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.002s
I20260809 07:32:28.938892 31244 raft_consensus.cc:348] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.939668 31244 raft_consensus.cc:374] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.939977 31244 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.940711 31244 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.940203 30741 catalog_manager.cc:5495] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:28.941115 31243 raft_consensus.cc:348] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.941762 31243 raft_consensus.cc:374] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.942018 31243 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.943063 31243 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.945568 31245 tablet_bootstrap.cc:492] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:28.946012 31245 ts_tablet_manager.cc:1397] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.174s user 0.015s sys 0.006s
I20260809 07:32:28.952915 31245 raft_consensus.cc:348] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.953545 31245 raft_consensus.cc:374] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:28.953801 31245 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:28.954380 31245 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.956322 31245 ts_tablet_manager.cc:1428] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260809 07:32:28.957077 31245 tablet_bootstrap.cc:492] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:28.958606 31243 ts_tablet_manager.cc:1428] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260809 07:32:28.959127 31244 ts_tablet_manager.cc:1428] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260809 07:32:28.958665 30742 catalog_manager.cc:5495] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:28.959959 31244 tablet_bootstrap.cc:492] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:28.960088 31243 tablet_bootstrap.cc:492] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:28.967499 30744 catalog_manager.cc:5495] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:28.972702 31245 tablet_bootstrap.cc:654] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.976274 30909 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:28.976886 31332 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:28.977445 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:28.977342 31332 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.978538 31297 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:28.978998 31297 raft_consensus.cc:686] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:28.979208 31332 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:28.979557 31244 tablet_bootstrap.cc:654] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:28.979777 31297 consensus_queue.cc:237] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:28.980774 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:28.984421 31006 leader_election.cc:341] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.985965 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:28.987267 31005 leader_election.cc:341] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:28.987668 31005 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:28.988359 31326 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:28.992491 31243 tablet_bootstrap.cc:654] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.000731 31245 tablet_bootstrap.cc:492] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:29.001181 31245 ts_tablet_manager.cc:1397] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.044s user 0.011s sys 0.019s
I20260809 07:32:29.003804 31245 raft_consensus.cc:348] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.004515 31245 raft_consensus.cc:374] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.004769 31245 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.005373 31245 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.007340 31245 ts_tablet_manager.cc:1428] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:32:29.008162 31245 tablet_bootstrap.cc:492] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:29.008312 31243 tablet_bootstrap.cc:492] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:29.008750 31243 ts_tablet_manager.cc:1397] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.007s
I20260809 07:32:29.012825 31345 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.014010 31345 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.016055 31244 tablet_bootstrap.cc:492] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.016500 31244 ts_tablet_manager.cc:1397] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.057s user 0.004s sys 0.011s
I20260809 07:32:29.016417 31345 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.018428 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.018867 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.020484 31057 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
W20260809 07:32:29.020550 30853 leader_election.cc:341] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.021128 31252 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.021658 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:29.022765 31345 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:29.023114 31345 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:29.023444 31345 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.023487 31243 raft_consensus.cc:348] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.024317 31243 raft_consensus.cc:374] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.019686 31244 raft_consensus.cc:348] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.024674 31244 raft_consensus.cc:374] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.024966 31244 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.024442 30744 catalog_manager.cc:5495] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:29.025631 31243 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.026278 31243 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.027581 31244 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.033453 31243 ts_tablet_manager.cc:1428] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20260809 07:32:29.038975 31345 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.040917 31345 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.041929 31243 tablet_bootstrap.cc:492] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:29.044620 31244 ts_tablet_manager.cc:1428] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260809 07:32:29.045502 31244 tablet_bootstrap.cc:492] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.041966 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:29.047317 30853 leader_election.cc:341] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.041967 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:29.050518 31245 tablet_bootstrap.cc:654] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.054994 31252 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.056154 30906 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.050611 31057 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.063335 31363 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.066493 31363 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.067361 31358 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.068466 31363 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.070140 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:29.071067 30853 consensus_peers.cc:487] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:29.072643 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.074916 31055 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:29.081794 31243 tablet_bootstrap.cc:654] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:29.074106 31120 leader_election.cc:341] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.089044 31244 tablet_bootstrap.cc:654] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.096866 31245 tablet_bootstrap.cc:492] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:29.097400 31245 ts_tablet_manager.cc:1397] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.089s user 0.005s sys 0.013s
I20260809 07:32:29.099335 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.100522 31252 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:29.100886 31252 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:29.101501 31252 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.104395 31245 raft_consensus.cc:348] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.105075 31245 raft_consensus.cc:374] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.105329 31245 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.105901 31245 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.100963 31057 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:29.112479 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:29.113526 31358 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:29.113979 31358 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:29.114800 31358 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.120904 31252 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.126945 31297 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.130136 31252 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.133374 31245 ts_tablet_manager.cc:1428] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.036s user 0.006s sys 0.001s
I20260809 07:32:29.134814 31245 tablet_bootstrap.cc:492] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:29.141894 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:29.142578 31057 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.144411 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:29.145798 31120 leader_election.cc:341] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.161629 31057 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:29.167030 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.168840 31363 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:29.169353 31363 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:29.167201 31345 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.172020 31363 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.172668 31345 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.175817 31243 tablet_bootstrap.cc:492] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:29.176301 31243 ts_tablet_manager.cc:1397] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.135s user 0.014s sys 0.007s
I20260809 07:32:29.181198 31345 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.182128 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.182976 31243 raft_consensus.cc:348] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.183369 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.183759 31243 raft_consensus.cc:374] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.184118 31243 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.184741 31243 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:29.185150 30852 leader_election.cc:341] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.188776 31181 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:29.190289 31243 ts_tablet_manager.cc:1428] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260809 07:32:29.191195 31243 tablet_bootstrap.cc:492] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:29.191331 30853 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.192586 31345 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:29.192943 31345 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:29.194204 31345 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.195757 30744 catalog_manager.cc:5495] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:29.201687 31345 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.204270 30742 catalog_manager.cc:5495] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:29.205216 31326 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.206578 31243 tablet_bootstrap.cc:654] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.207232 31244 tablet_bootstrap.cc:492] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.207664 31244 ts_tablet_manager.cc:1397] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.162s user 0.008s sys 0.012s
I20260809 07:32:29.212838 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:29.225389 31345 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.226775 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:29.227415 31182 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:29.229393 30852 leader_election.cc:341] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.234207 31182 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:29.235430 30853 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.240135 31345 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:29.240629 31345 raft_consensus.cc:686] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:29.241426 31345 consensus_queue.cc:237] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.244336 31253 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.244594 31244 raft_consensus.cc:348] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.245338 31244 raft_consensus.cc:374] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.245612 31244 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.247001 31244 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.246032 31253 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.249779 31244 ts_tablet_manager.cc:1428] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.042s user 0.006s sys 0.015s
I20260809 07:32:29.250502 31253 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.250548 31354 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.251116 31354 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.252557 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.253342 31354 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.253439 31059 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:29.255136 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.256578 31244 tablet_bootstrap.cc:492] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.258766 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.259505 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.260071 31059 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:29.258805 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:29.268687 31358 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:29.269194 31358 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:32:29.265085 30853 leader_election.cc:341] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.269527 31358 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:29.272590 30853 leader_election.cc:341] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.275610 30852 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:29.277148 31354 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:29.277485 31354 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:29.277729 31354 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.282555 31245 tablet_bootstrap.cc:654] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.286083 31243 tablet_bootstrap.cc:492] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:29.286585 31243 ts_tablet_manager.cc:1397] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.096s user 0.006s sys 0.022s
I20260809 07:32:29.290904 31243 raft_consensus.cc:348] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.291603 31243 raft_consensus.cc:374] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.291857 31243 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.292430 31243 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.293442 30745 catalog_manager.cc:5495] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:29.294401 31363 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.294862 31363 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.296660 31363 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.298178 31243 ts_tablet_manager.cc:1428] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260809 07:32:29.298076 31354 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.303069 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:29.303680 31354 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.303566 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:29.304237 31059 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.304512 31244 tablet_bootstrap.cc:654] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.306457 31358 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:29.307251 30853 leader_election.cc:341] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.309173 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:29.310520 30853 leader_election.cc:341] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.310751 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:29.311365 31057 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.314682 31358 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.315459 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.316246 30911 raft_consensus.cc:2379] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:29.319201 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.321091 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.322458 31252 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.323661 31245 tablet_bootstrap.cc:492] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:29.324103 31245 ts_tablet_manager.cc:1397] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.190s user 0.020s sys 0.010s
I20260809 07:32:29.326476 31245 raft_consensus.cc:348] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.327126 31245 raft_consensus.cc:374] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.327386 31245 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.327950 31245 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.330394 31245 ts_tablet_manager.cc:1428] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:32:29.331919 31057 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:29.333053 31252 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.333189 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:29.335656 31358 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:29.336274 31358 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:29.337960 31358 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.341578 31245 tablet_bootstrap.cc:492] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:29.341449 31059 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:29.343372 30852 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:29.344218 31354 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:29.344650 31354 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:29.344715 31491 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.345376 31491 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.345466 31354 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.346983 31245 tablet_bootstrap.cc:654] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.347856 31491 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.349292 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.350061 31182 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:29.353461 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.354754 31005 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.356021 31332 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.366730 31252 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.375345 31363 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.376051 31363 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.378058 31363 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.379640 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.384176 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.385949 31244 tablet_bootstrap.cc:492] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.386590 31243 tablet_bootstrap.cc:492] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:29.367260 31252 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.385995 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.387181 31244 ts_tablet_manager.cc:1397] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.131s user 0.026s sys 0.003s
I20260809 07:32:29.387791 31363 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.389834 31244 raft_consensus.cc:348] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.390511 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.390784 31244 raft_consensus.cc:374] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.391211 31244 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
W20260809 07:32:29.391875 31120 leader_election.cc:341] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.391996 31244 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.394055 31332 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.394639 31244 ts_tablet_manager.cc:1428] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
W20260809 07:32:29.398259 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:29.398662 31252 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.400943 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.401381 31244 tablet_bootstrap.cc:492] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.408217 31244 tablet_bootstrap.cc:654] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.408674 31243 tablet_bootstrap.cc:654] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:29.417377 31120 leader_election.cc:341] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.417977 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.418888 31244 tablet_bootstrap.cc:492] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.418069 30745 catalog_manager.cc:5495] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:29.419314 31244 ts_tablet_manager.cc:1397] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
W20260809 07:32:29.419319 31120 consensus_peers.cc:487] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 -> Peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Couldn't send request to peer 2c9de56c3fa3487c9dac990675c34be5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:29.421299 31252 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.424928 31244 raft_consensus.cc:348] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.425590 31244 raft_consensus.cc:374] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.425858 31244 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.426446 31244 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.430841 31354 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.433960 31244 ts_tablet_manager.cc:1428] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260809 07:32:29.434813 31244 tablet_bootstrap.cc:492] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.439917 31244 tablet_bootstrap.cc:654] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.441483 31354 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.441931 31354 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.447086 31363 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.450438 31354 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.451263 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:29.452662 30852 leader_election.cc:341] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.453415 31363 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.459020 31244 tablet_bootstrap.cc:492] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.458730 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.459467 31244 ts_tablet_manager.cc:1397] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.025s user 0.014s sys 0.009s
W20260809 07:32:29.460208 30853 leader_election.cc:341] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.460667 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.461802 31244 raft_consensus.cc:348] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.462494 31244 raft_consensus.cc:374] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.462749 31244 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.461673 31354 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.463326 31244 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.465302 31244 ts_tablet_manager.cc:1428] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260809 07:32:29.466146 31244 tablet_bootstrap.cc:492] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.471755 31244 tablet_bootstrap.cc:654] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.479738 31345 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.439885 30742 catalog_manager.cc:5495] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:29.486261 31245 tablet_bootstrap.cc:492] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:29.486809 31245 ts_tablet_manager.cc:1397] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.145s user 0.009s sys 0.002s
I20260809 07:32:29.489298 31245 raft_consensus.cc:348] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.489972 31245 raft_consensus.cc:374] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.503720 31243 tablet_bootstrap.cc:492] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:29.504128 31243 ts_tablet_manager.cc:1397] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.118s user 0.000s sys 0.011s
I20260809 07:32:29.490224 31245 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.505151 31245 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.507306 31245 ts_tablet_manager.cc:1428] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20260809 07:32:29.508378 31245 tablet_bootstrap.cc:492] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:29.509637 31243 raft_consensus.cc:348] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.510427 31243 raft_consensus.cc:374] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.510632 31243 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.511168 31243 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.521440 31243 ts_tablet_manager.cc:1428] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.017s user 0.004s sys 0.002s
I20260809 07:32:29.522296 31243 tablet_bootstrap.cc:492] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:29.531418 31181 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.532145 31363 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.553171 31243 tablet_bootstrap.cc:654] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.555936 31245 tablet_bootstrap.cc:654] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.557678 31491 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.558617 31244 tablet_bootstrap.cc:492] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.559054 31244 ts_tablet_manager.cc:1397] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.093s user 0.020s sys 0.028s
I20260809 07:32:29.561492 31244 raft_consensus.cc:348] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.562137 31244 raft_consensus.cc:374] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.563364 31244 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.563951 31244 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.565937 31244 ts_tablet_manager.cc:1428] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260809 07:32:29.566857 31244 tablet_bootstrap.cc:492] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.572105 31244 tablet_bootstrap.cc:654] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.572736 31245 tablet_bootstrap.cc:492] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:29.573328 31245 ts_tablet_manager.cc:1397] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.000s
I20260809 07:32:29.576058 31245 raft_consensus.cc:348] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.576794 31245 raft_consensus.cc:374] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.577108 31245 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.577768 31245 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.580173 31245 ts_tablet_manager.cc:1428] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:32:29.581063 31245 tablet_bootstrap.cc:492] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:29.583248 31244 tablet_bootstrap.cc:492] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.583694 31244 ts_tablet_manager.cc:1397] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260809 07:32:29.584597 31491 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.585598 31243 tablet_bootstrap.cc:492] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:29.586030 31243 ts_tablet_manager.cc:1397] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.064s user 0.006s sys 0.005s
I20260809 07:32:29.588474 31243 raft_consensus.cc:348] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.589064 31252 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.589118 31243 raft_consensus.cc:374] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.589564 31243 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.589573 31252 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.590258 31243 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.592046 31252 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.592239 31243 ts_tablet_manager.cc:1428] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:32:29.593079 31243 tablet_bootstrap.cc:492] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:29.595458 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.586014 31244 raft_consensus.cc:348] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.597893 31244 raft_consensus.cc:374] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.598167 31244 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.599776 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.600649 31058 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:29.598779 31244 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.604423 31244 ts_tablet_manager.cc:1428] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260809 07:32:29.605321 31345 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.605790 31243 tablet_bootstrap.cc:654] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.606932 30906 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.606932 31244 tablet_bootstrap.cc:492] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.615417 31244 tablet_bootstrap.cc:654] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.615180 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:29.618238 31252 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.624495 31252 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.625771 31252 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.628983 31245 tablet_bootstrap.cc:654] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.637768 31332 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.638273 31332 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.645058 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.646656 31332 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:29.649392 30853 consensus_peers.cc:487] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:29.651863 31332 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.652593 31059 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.652392 31332 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.661589 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:29.662998 31005 leader_election.cc:341] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.665961 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.666874 31177 raft_consensus.cc:2379] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:29.669265 31332 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.669468 31354 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:29.672120 31243 tablet_bootstrap.cc:492] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:29.672559 31243 ts_tablet_manager.cc:1397] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.080s user 0.012s sys 0.005s
I20260809 07:32:29.676055 31243 raft_consensus.cc:348] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.677083 31243 raft_consensus.cc:374] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.677345 31243 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.675874 31363 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.678484 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.678777 31363 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.677973 31243 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.679755 31244 tablet_bootstrap.cc:492] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:29.680266 31244 ts_tablet_manager.cc:1397] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.010s
I20260809 07:32:29.673712 31006 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.699187 31243 ts_tablet_manager.cc:1428] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.026s user 0.006s sys 0.001s
W20260809 07:32:29.682580 31006 leader_election.cc:341] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.700260 31243 tablet_bootstrap.cc:492] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:29.704490 31006 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.707401 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.711084 31490 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.711943 31297 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.711680 31490 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.715277 31490 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:29.715584 31120 leader_election.cc:341] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.707417 31363 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.720525 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.723263 31332 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.725076 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.721347 31058 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:29.726758 31244 raft_consensus.cc:348] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.727348 31244 raft_consensus.cc:374] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.727556 31244 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.728031 31244 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.732012 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:29.735737 31005 leader_election.cc:341] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.736321 31245 tablet_bootstrap.cc:492] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:29.736773 31245 ts_tablet_manager.cc:1397] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.156s user 0.005s sys 0.007s
W20260809 07:32:29.742458 31006 leader_election.cc:341] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.744869 31006 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.746280 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.747967 31252 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260809 07:32:29.753383 31345 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:29.755040 31245 raft_consensus.cc:348] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.755760 31245 raft_consensus.cc:374] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:29.756014 31245 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:29.747816 31490 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.757906 31354 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:29.760679 31245 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.761858 31252 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:29.762414 31252 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:29.762689 31252 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.776927 31244 ts_tablet_manager.cc:1428] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.096s user 0.004s sys 0.002s
I20260809 07:32:29.777894 31244 tablet_bootstrap.cc:492] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:29.780351 31326 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.781217 31326 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.786444 31182 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:29.802431 31245 ts_tablet_manager.cc:1428] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.065s user 0.008s sys 0.002s
I20260809 07:32:29.803748 31245 tablet_bootstrap.cc:492] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
W20260809 07:32:29.805641 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 69442.2us
I20260809 07:32:29.818848 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:29.827693 31005 leader_election.cc:341] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.814967 31326 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.831053 31243 tablet_bootstrap.cc:654] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.885249 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.894820 31294 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.895558 31294 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.898487 31332 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.899241 31332 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.900628 31244 tablet_bootstrap.cc:654] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.903347 31332 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.885773 31326 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:29.910951 31006 leader_election.cc:341] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.911671 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.916469 31245 tablet_bootstrap.cc:654] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:29.911805 31006 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:29.923570 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.901805 31252 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.905853 31326 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.926378 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.923617 31490 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:29.928465 31252 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.930568 31358 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.931149 31358 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.928849 31294 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:29.932348 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:29.934695 31354 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:29.935400 31354 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:29.935603 31345 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 07:32:29.936956 31120 leader_election.cc:341] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.938967 31326 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.939580 31354 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:29.934036 31006 leader_election.cc:341] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:29.942062 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.944350 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:29.945277 31058 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:30.134706 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.955868 31345 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.181s
I20260809 07:32:29.934243 31358 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:29.964195 31288 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:30.128937 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 110029us
I20260809 07:32:29.948073 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.094704 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:29.955507 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:29.992914 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:29.993505 31006 leader_election.cc:341] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.145143 31288 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.155644 30852 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:30.157883 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:30.166306 30906 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:30.167757 31243 tablet_bootstrap.cc:492] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:30.168227 31243 ts_tablet_manager.cc:1397] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.468s user 0.012s sys 0.030s
I20260809 07:32:30.153826 31288 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:30.172986 31243 raft_consensus.cc:348] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.173745 31243 raft_consensus.cc:374] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.173998 31243 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.170622 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:30.154400 30911 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
W20260809 07:32:30.178630 30853 leader_election.cc:341] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.180544 31243 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:30.155369 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 185464us
I20260809 07:32:30.171121 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:30.186241 31243 ts_tablet_manager.cc:1428] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260809 07:32:30.182224 31354 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:30.190963 31243 tablet_bootstrap.cc:492] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:30.191198 31054 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:30.182915 30852 leader_election.cc:341] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.194861 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:30.195565 30853 leader_election.cc:341] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.201964 31245 tablet_bootstrap.cc:492] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:30.202431 31245 ts_tablet_manager.cc:1397] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.399s user 0.027s sys 0.013s
I20260809 07:32:30.202674 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.203426 31345 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.204813 31245 raft_consensus.cc:348] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.205466 31245 raft_consensus.cc:374] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.205701 31245 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.185132 31543 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260809 07:32:30.206439 31537 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:30.206261 31245 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.206817 31537 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:30.359274 31244 tablet_bootstrap.cc:492] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:30.359830 31244 ts_tablet_manager.cc:1397] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.582s user 0.022s sys 0.054s
I20260809 07:32:30.341987 31054 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:30.361853 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:29.965224 31005 leader_election.cc:341] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.367194 31245 ts_tablet_manager.cc:1428] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.164s user 0.006s sys 0.000s
I20260809 07:32:30.368248 31245 tablet_bootstrap.cc:492] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:30.372005 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:30.360832 31537 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.286873 31354 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:32:30.369293 31534 log.cc:924] Time spent T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.147s user 0.000s sys 0.031s
I20260809 07:32:30.378459 31245 tablet_bootstrap.cc:654] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:30.380806 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 415641us
I20260809 07:32:30.385807 31244 raft_consensus.cc:348] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.386518 31244 raft_consensus.cc:374] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.386762 31244 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.387216 31244 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.384061 31558 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.394308 31243 tablet_bootstrap.cc:654] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.394766 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:30.395749 31120 leader_election.cc:341] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.379114 31354 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.397908 31244 ts_tablet_manager.cc:1428] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260809 07:32:30.398644 31363 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:30.398741 31244 tablet_bootstrap.cc:492] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:30.401029 31554 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:30.401569 31554 raft_consensus.cc:686] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:30.420096 31554 consensus_queue.cc:237] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.429225 31363 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.429811 31363 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.433542 31363 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:30.437390 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:30.448200 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:30.449728 31490 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:30.450091 31490 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:30.450438 31490 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.453125 31354 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.455009 31354 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.456148 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:30.457468 30853 leader_election.cc:341] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.459591 31537 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.461614 31537 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:30.467099 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:30.468287 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:30.468902 30911 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.469696 30745 catalog_manager.cc:5495] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:30.478080 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.479137 31244 tablet_bootstrap.cc:654] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.483523 30911 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:30.484637 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:30.485476 31537 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:30.485947 31537 raft_consensus.cc:686] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:30.486816 31490 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.486968 31537 consensus_queue.cc:237] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.486811 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:30.492173 31054 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:30.493672 31490 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.495292 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:30.495491 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:30.496842 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.499311 30906 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:30.498646 31363 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:32:30.503103 31120 leader_election.cc:341] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:30.502430 31006 leader_election.cc:341] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.508248 30852 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.514122 31245 tablet_bootstrap.cc:492] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:30.515283 31245 ts_tablet_manager.cc:1397] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.147s user 0.007s sys 0.009s
I20260809 07:32:30.516047 31006 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:30.519902 31490 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.521136 31243 tablet_bootstrap.cc:492] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:30.521505 31243 ts_tablet_manager.cc:1397] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.333s user 0.016s sys 0.013s
W20260809 07:32:30.522765 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:30.523088 31358 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.555794 31245 raft_consensus.cc:348] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.556553 31245 raft_consensus.cc:374] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.556825 31245 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.557531 31245 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.559221 31245 ts_tablet_manager.cc:1428] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.044s user 0.006s sys 0.003s
I20260809 07:32:30.559962 31245 tablet_bootstrap.cc:492] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:30.563482 31243 raft_consensus.cc:348] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.564328 31243 raft_consensus.cc:374] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.564641 31243 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.565223 31244 tablet_bootstrap.cc:492] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:30.565696 31244 ts_tablet_manager.cc:1397] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.167s user 0.012s sys 0.025s
I20260809 07:32:30.565346 31243 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.566650 31245 tablet_bootstrap.cc:654] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.573495 31244 raft_consensus.cc:348] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.574240 31244 raft_consensus.cc:374] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.580045 31244 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.580806 31244 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.581585 31243 ts_tablet_manager.cc:1428] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.047s user 0.005s sys 0.001s
I20260809 07:32:30.582624 31243 tablet_bootstrap.cc:492] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:30.586391 31244 ts_tablet_manager.cc:1428] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.020s user 0.005s sys 0.004s
I20260809 07:32:30.589962 31244 tablet_bootstrap.cc:492] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:30.591477 31245 tablet_bootstrap.cc:492] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:30.592052 31245 ts_tablet_manager.cc:1397] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.006s
I20260809 07:32:30.594635 31245 raft_consensus.cc:348] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.595291 31245 raft_consensus.cc:374] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.595543 31245 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.596184 31245 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.597146 31243 tablet_bootstrap.cc:654] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.605079 31244 tablet_bootstrap.cc:654] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.610581 31245 ts_tablet_manager.cc:1428] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260809 07:32:30.611248 30906 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.611477 31245 tablet_bootstrap.cc:492] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:30.617161 31245 tablet_bootstrap.cc:654] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.638290 31490 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.637603 30744 catalog_manager.cc:5495] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:30.638863 31490 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.640803 31490 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.649537 31243 tablet_bootstrap.cc:492] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:30.649430 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:30.650080 31243 ts_tablet_manager.cc:1397] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.009s
I20260809 07:32:30.650174 31177 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:30.651809 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:30.653604 31005 leader_election.cc:341] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.653682 31243 raft_consensus.cc:348] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.654414 31243 raft_consensus.cc:374] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.654706 31243 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.655244 31243 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.658790 31006 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:30.659559 31490 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:30.663831 31490 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:30.664175 31490 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.670423 31490 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.660455 31243 ts_tablet_manager.cc:1428] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260809 07:32:30.672859 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:30.673949 31005 leader_election.cc:341] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.674973 31490 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.675392 31243 tablet_bootstrap.cc:492] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:30.676034 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:30.677416 31177 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.686389 31358 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:30.686949 31358 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.687829 31244 tablet_bootstrap.cc:492] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:30.688287 31244 ts_tablet_manager.cc:1397] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.099s user 0.013s sys 0.008s
I20260809 07:32:30.695297 31244 raft_consensus.cc:348] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.695789 31244 raft_consensus.cc:374] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.695994 31244 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.697214 31244 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.698292 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.699822 31245 tablet_bootstrap.cc:492] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:30.700275 31358 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.710956 31245 ts_tablet_manager.cc:1397] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.100s user 0.016s sys 0.020s
I20260809 07:32:30.712141 31244 ts_tablet_manager.cc:1428] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.024s user 0.004s sys 0.003s
I20260809 07:32:30.713055 31244 tablet_bootstrap.cc:492] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:30.713359 31245 raft_consensus.cc:348] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.701896 31182 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.716128 31358 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.702992 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:30.717011 31358 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.718595 30853 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.719374 31358 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.719950 31358 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:30.720367 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:30.723093 30853 leader_election.cc:341] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.723404 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.724404 31054 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:30.725319 31243 tablet_bootstrap.cc:654] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.706233 31177 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:30.714003 31245 raft_consensus.cc:374] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.726076 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:30.736729 31245 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.737501 31245 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.745571 31244 tablet_bootstrap.cc:654] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.740932 31545 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:30.746299 31245 ts_tablet_manager.cc:1428] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260809 07:32:30.746044 31545 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:30.747210 31545 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.749459 31245 tablet_bootstrap.cc:492] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:30.758862 31006 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:30.770984 31557 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:30.771859 31557 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:30.780999 31245 tablet_bootstrap.cc:654] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.773458 31557 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.787732 31543 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.787770 31243 tablet_bootstrap.cc:492] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:30.792729 31243 ts_tablet_manager.cc:1397] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.118s user 0.004s sys 0.026s
I20260809 07:32:30.793578 31543 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.795578 31543 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.795399 31243 raft_consensus.cc:348] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.796083 31243 raft_consensus.cc:374] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.796346 31243 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.797778 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.796936 31243 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.799350 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:30.800074 31177 raft_consensus.cc:2379] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:30.800170 31545 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.806481 30853 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.808015 30742 catalog_manager.cc:5495] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:30.809836 31543 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.800439 31243 ts_tablet_manager.cc:1428] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260809 07:32:30.811944 31243 tablet_bootstrap.cc:492] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:30.813496 31244 tablet_bootstrap.cc:492] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:30.813903 31244 ts_tablet_manager.cc:1397] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.101s user 0.010s sys 0.007s
I20260809 07:32:30.816823 31543 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.816587 31244 raft_consensus.cc:348] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.817358 31244 raft_consensus.cc:374] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.817637 31244 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.818929 31244 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.820184 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:30.822669 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:30.825898 31245 tablet_bootstrap.cc:492] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:30.836805 31245 ts_tablet_manager.cc:1397] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.088s user 0.005s sys 0.007s
W20260809 07:32:30.837065 30853 leader_election.cc:341] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.836249 31055 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.839365 31245 raft_consensus.cc:348] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.840035 31245 raft_consensus.cc:374] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.840286 31245 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.840842 31245 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.832578 31244 ts_tablet_manager.cc:1428] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.018s user 0.004s sys 0.001s
I20260809 07:32:30.843199 31244 tablet_bootstrap.cc:492] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:30.847057 31055 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:30.825132 31243 tablet_bootstrap.cc:654] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.819876 31545 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.851044 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:30.851709 31245 ts_tablet_manager.cc:1428] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260809 07:32:30.852674 31245 tablet_bootstrap.cc:492] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:30.853138 31543 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:30.853600 31543 raft_consensus.cc:686] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:30.854382 31543 consensus_queue.cc:237] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.865952 31244 tablet_bootstrap.cc:654] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.871374 31245 tablet_bootstrap.cc:654] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:30.874075 30745 catalog_manager.cc:5495] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:30.885823 31543 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.886368 31543 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.889868 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.901099 31243 tablet_bootstrap.cc:492] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:30.901750 31243 ts_tablet_manager.cc:1397] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.090s user 0.017s sys 0.008s
I20260809 07:32:30.901957 31055 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:30.890380 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:30.893958 31554 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260809 07:32:30.905689 30853 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.906718 31545 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.915707 31243 raft_consensus.cc:348] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.916922 31243 raft_consensus.cc:374] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.917188 31243 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.917800 31243 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.922619 31243 ts_tablet_manager.cc:1428] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.020s user 0.007s sys 0.001s
I20260809 07:32:30.890494 31543 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:30.924000 31243 tablet_bootstrap.cc:492] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:30.929752 31557 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.942572 31244 tablet_bootstrap.cc:492] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:30.943066 31244 ts_tablet_manager.cc:1397] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.100s user 0.011s sys 0.009s
I20260809 07:32:30.932361 31557 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.947407 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:30.947750 31557 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:30.949507 31005 leader_election.cc:341] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.953421 31537 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:30.954736 31545 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.953459 31245 tablet_bootstrap.cc:492] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:30.954193 31537 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.955672 31245 ts_tablet_manager.cc:1397] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.103s user 0.009s sys 0.015s
I20260809 07:32:30.956459 31244 raft_consensus.cc:348] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.957127 31244 raft_consensus.cc:374] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.957376 31244 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.950990 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:30.959184 31182 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:30.961504 31006 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:30.961748 31556 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:30.962770 31245 raft_consensus.cc:348] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.965898 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:30.966450 31245 raft_consensus.cc:374] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:30.966260 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:30.966897 31245 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:30.958127 31244 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.964003 31537 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:30.967778 31120 leader_election.cc:341] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.966146 31243 tablet_bootstrap.cc:654] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:30.970252 31120 leader_election.cc:341] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:30.967494 31557 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:30.970816 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:30.971081 31557 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:30.971596 31557 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:30.972332 31244 ts_tablet_manager.cc:1428] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260809 07:32:30.972210 31363 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:30.973148 31244 tablet_bootstrap.cc:492] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:30.967531 31245 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:30.984078 31245 ts_tablet_manager.cc:1428] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.028s user 0.000s sys 0.005s
I20260809 07:32:30.984932 31245 tablet_bootstrap.cc:492] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:31.014452 31245 tablet_bootstrap.cc:654] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.039742 31243 tablet_bootstrap.cc:492] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:31.040273 31243 ts_tablet_manager.cc:1397] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.116s user 0.019s sys 0.006s
I20260809 07:32:31.048229 31558 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.049031 31558 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.051291 31557 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.049085 31363 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.058136 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.058830 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.059149 30904 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:31.059789 31182 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:31.061338 31005 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:31.062116 31558 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.063356 31490 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.063966 31490 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.067034 31490 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.067752 31244 tablet_bootstrap.cc:654] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.072563 31558 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.073377 31558 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.078516 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.081930 31243 raft_consensus.cc:348] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.091951 31243 raft_consensus.cc:374] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.092172 31243 raft_consensus.cc:729] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.078717 31557 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.090476 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.091030 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:31.094717 31182 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.102864 31006 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.092605 31243 consensus_queue.cc:260] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.102574 31490 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.110461 31557 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.119840 31055 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.120631 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:31.121520 30911 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.125933 31243 ts_tablet_manager.cc:1428] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.085s user 0.006s sys 0.002s
I20260809 07:32:31.126969 31243 tablet_bootstrap.cc:492] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:31.082367 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:31.130739 31005 leader_election.cc:341] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.091217 31558 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.136837 31490 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.141453 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:31.146907 31177 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.154120 31363 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.157732 30911 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:31.166746 31005 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:31.160001 31244 tablet_bootstrap.cc:492] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.168691 31244 ts_tablet_manager.cc:1397] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.196s user 0.026s sys 0.013s
I20260809 07:32:31.168445 31490 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:31.169364 31490 raft_consensus.cc:686] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:31.172930 31244 raft_consensus.cc:348] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.174468 31245 tablet_bootstrap.cc:492] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.174932 31245 ts_tablet_manager.cc:1397] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.190s user 0.023s sys 0.034s
I20260809 07:32:31.174609 31244 raft_consensus.cc:374] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.176751 31182 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:31.179802 31363 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.195773 31244 raft_consensus.cc:729] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.196017 31363 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.184624 31490 consensus_queue.cc:237] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.197172 31244 consensus_queue.cc:260] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.198585 31363 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.200076 31244 ts_tablet_manager.cc:1428] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.031s user 0.001s sys 0.005s
I20260809 07:32:31.200588 31006 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.200969 31244 tablet_bootstrap.cc:492] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:31.201820 31558 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:31.202565 31558 raft_consensus.cc:686] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:31.203928 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.204104 31558 consensus_queue.cc:237] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.189685 31177 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:31.190693 31245 raft_consensus.cc:348] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.215088 31245 raft_consensus.cc:374] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.215370 31245 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.195469 31243 tablet_bootstrap.cc:654] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:31.207595 31120 leader_election.cc:341] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.209224 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.217315 31245 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.214551 31244 tablet_bootstrap.cc:654] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.219532 31054 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:31.224316 31245 ts_tablet_manager.cc:1428] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260809 07:32:31.224778 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.225242 31245 tablet_bootstrap.cc:492] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:31.225960 31556 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.226339 31556 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.226655 31556 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.226887 30742 catalog_manager.cc:5495] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:31.238857 31245 tablet_bootstrap.cc:654] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.240790 31554 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.241290 31554 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.243292 31554 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.246383 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.246917 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:31.251067 31120 leader_election.cc:341] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.248678 31055 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:31.266094 31542 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.267580 31542 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.268981 31244 tablet_bootstrap.cc:492] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.268872 31556 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.270818 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.271853 31556 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.273034 31542 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.273808 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.274544 31244 ts_tablet_manager.cc:1397] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.074s user 0.012s sys 0.013s
I20260809 07:32:31.274462 31054 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:31.272505 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.276850 30744 catalog_manager.cc:5495] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:32:31.280268 30853 leader_election.cc:341] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.280555 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:31.281960 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:31.282521 31054 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:31.287276 31120 leader_election.cc:341] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.290063 30852 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:31.290382 31244 raft_consensus.cc:348] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.291052 31244 raft_consensus.cc:374] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.291312 31244 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.292157 31543 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.291970 31244 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.292570 31543 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.292927 31543 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.286859 31556 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.293856 31556 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.293987 31244 ts_tablet_manager.cc:1428] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260809 07:32:31.294349 31556 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.294958 31244 tablet_bootstrap.cc:492] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:31.298358 31542 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.298870 31542 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.305022 31243 tablet_bootstrap.cc:492] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:31.305466 31243 ts_tablet_manager.cc:1397] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.179s user 0.016s sys 0.025s
I20260809 07:32:31.306182 31542 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.306483 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.308144 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:31.308463 30853 leader_election.cc:341] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:31.309306 30852 leader_election.cc:341] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.308033 31243 raft_consensus.cc:348] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.309886 31243 raft_consensus.cc:374] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.310642 31243 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.311165 31243 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.310505 31558 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.313318 31543 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.311429 30852 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:31.317901 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:31.318471 31055 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.321246 31054 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:31.323698 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.325163 31243 ts_tablet_manager.cc:1428] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
I20260809 07:32:31.326062 31243 tablet_bootstrap.cc:492] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:31.329708 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:31.315259 31543 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.332094 31245 tablet_bootstrap.cc:492] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.332577 31245 ts_tablet_manager.cc:1397] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.108s user 0.015s sys 0.012s
I20260809 07:32:31.332609 31554 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 07:32:31.338500 30853 leader_election.cc:341] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.333109 31554 raft_consensus.cc:686] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:31.333377 31542 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.339669 31554 consensus_queue.cc:237] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.324764 31055 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:31.342501 31245 raft_consensus.cc:348] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.343812 31245 raft_consensus.cc:374] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.344096 31245 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.349587 30852 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:31.350543 31542 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:31.350997 31542 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:31.351718 31542 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.314996 31556 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.355360 31556 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.357513 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:31.359019 31244 tablet_bootstrap.cc:654] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.349493 31245 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:31.361334 31120 leader_election.cc:341] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.362931 31558 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.370730 31558 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.375903 31243 tablet_bootstrap.cc:654] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.376224 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:31.378530 31005 leader_election.cc:341] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.381476 31245 ts_tablet_manager.cc:1428] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.049s user 0.007s sys 0.001s
I20260809 07:32:31.386879 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:31.387444 31054 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.387898 31245 tablet_bootstrap.cc:492] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:31.389007 30742 catalog_manager.cc:5495] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:31.393788 31054 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:31.386373 30744 catalog_manager.cc:5495] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:31.400200 31543 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.400679 31543 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.402611 31543 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.403348 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:31.404984 30853 consensus_peers.cc:487] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:31.405030 31558 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.405982 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.406039 31557 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.406787 31181 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:31.408227 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:31.409314 31545 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.413727 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:31.415349 31006 leader_election.cc:341] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.415750 31006 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.416697 31558 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.418870 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.420073 31556 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:31.420528 31556 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:31.421265 31556 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.425652 31545 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.435407 31245 tablet_bootstrap.cc:654] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.456394 31243 tablet_bootstrap.cc:492] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:31.456852 31243 ts_tablet_manager.cc:1397] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.131s user 0.013s sys 0.013s
I20260809 07:32:31.459224 31243 raft_consensus.cc:348] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.459892 31243 raft_consensus.cc:374] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.460121 31243 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.461050 31244 tablet_bootstrap.cc:492] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.460659 31243 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.461467 31244 ts_tablet_manager.cc:1397] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.167s user 0.010s sys 0.003s
I20260809 07:32:31.462865 31243 ts_tablet_manager.cc:1428] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:32:31.463740 31243 tablet_bootstrap.cc:492] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:31.463681 31244 raft_consensus.cc:348] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.466092 31244 raft_consensus.cc:374] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.466375 31244 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.468775 31244 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.470577 31243 tablet_bootstrap.cc:654] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.471050 31554 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.471516 31554 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.473347 31245 tablet_bootstrap.cc:492] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.473310 31554 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.473836 31245 ts_tablet_manager.cc:1397] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.086s user 0.008s sys 0.005s
I20260809 07:32:31.474480 31244 ts_tablet_manager.cc:1428] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260809 07:32:31.475524 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.475912 31244 tablet_bootstrap.cc:492] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:31.476107 31054 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:31.476020 31245 raft_consensus.cc:348] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.476740 31245 raft_consensus.cc:374] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.476989 31245 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.477303 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.479019 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:31.479831 31120 leader_election.cc:341] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.480624 31554 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.481017 31554 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.481360 31554 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.477635 31245 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.491014 31244 tablet_bootstrap.cc:654] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.491400 31542 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:31.492229 31245 ts_tablet_manager.cc:1428] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.018s user 0.005s sys 0.001s
I20260809 07:32:31.493206 31245 tablet_bootstrap.cc:492] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
W20260809 07:32:31.506811 31634 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.057s user 0.000s sys 0.001s
I20260809 07:32:31.505381 30742 catalog_manager.cc:5495] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:31.520715 31245 tablet_bootstrap.cc:654] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.524173 31243 tablet_bootstrap.cc:492] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:31.522924 31244 tablet_bootstrap.cc:492] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.538803 31244 ts_tablet_manager.cc:1397] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.063s user 0.026s sys 0.005s
I20260809 07:32:31.541330 31244 raft_consensus.cc:348] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.542012 31244 raft_consensus.cc:374] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.542295 31244 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.542928 31244 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.544983 31244 ts_tablet_manager.cc:1428] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260809 07:32:31.545872 31244 tablet_bootstrap.cc:492] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:31.533134 31245 tablet_bootstrap.cc:492] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.550063 31243 ts_tablet_manager.cc:1397] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.086s user 0.013s sys 0.007s
I20260809 07:32:31.550637 31245 ts_tablet_manager.cc:1397] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.004s
I20260809 07:32:31.555691 31243 raft_consensus.cc:348] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.556447 31243 raft_consensus.cc:374] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.556707 31243 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.557358 31243 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.564535 31244 tablet_bootstrap.cc:654] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.536511 31554 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.560490 31245 raft_consensus.cc:348] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.567055 31245 raft_consensus.cc:374] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.567322 31245 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.568040 31245 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.568827 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:31.570159 31120 leader_election.cc:341] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.570261 31245 ts_tablet_manager.cc:1428] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.019s user 0.000s sys 0.003s
I20260809 07:32:31.572639 31245 tablet_bootstrap.cc:492] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
W20260809 07:32:31.577128 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:31.582989 31245 tablet_bootstrap.cc:654] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.588680 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:31.595531 31054 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.589267 31554 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.594005 31243 ts_tablet_manager.cc:1428] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260809 07:32:31.602597 31054 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:31.604528 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.604423 31243 tablet_bootstrap.cc:492] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:31.605630 31554 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:31.606186 31554 raft_consensus.cc:686] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:31.607187 31554 consensus_queue.cc:237] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.616119 31245 tablet_bootstrap.cc:492] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.616784 31245 ts_tablet_manager.cc:1397] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.044s user 0.012s sys 0.000s
I20260809 07:32:31.618989 31244 tablet_bootstrap.cc:492] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.619465 31244 ts_tablet_manager.cc:1397] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.074s user 0.003s sys 0.011s
I20260809 07:32:31.621482 31243 tablet_bootstrap.cc:654] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.621328 31558 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.622171 31557 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.624531 31244 raft_consensus.cc:348] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.625231 31244 raft_consensus.cc:374] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.625554 31244 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.626736 31245 raft_consensus.cc:348] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.627439 31245 raft_consensus.cc:374] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.627741 31245 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.627514 31244 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.628435 31245 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.634785 31245 ts_tablet_manager.cc:1428] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260809 07:32:31.635705 31245 tablet_bootstrap.cc:492] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:31.638845 31557 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.639415 31557 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.644601 31557 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.645576 30742 catalog_manager.cc:5495] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:31.647854 31244 ts_tablet_manager.cc:1428] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.028s user 0.004s sys 0.004s
I20260809 07:32:31.647653 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.648247 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.648902 31244 tablet_bootstrap.cc:492] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
W20260809 07:32:31.650473 31006 leader_election.cc:341] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:31.660352 31005 leader_election.cc:341] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.661170 31005 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:31.664866 31558 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.669121 31245 tablet_bootstrap.cc:654] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.666247 31543 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.673637 31554 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.674676 31244 tablet_bootstrap.cc:654] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.674548 31554 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.680392 31554 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.674474 31543 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.685600 31243 tablet_bootstrap.cc:492] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:31.687124 31243 ts_tablet_manager.cc:1397] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.083s user 0.016s sys 0.015s
I20260809 07:32:31.689642 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.690302 31055 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:31.691598 31543 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.691758 30852 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:31.692735 31543 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.693118 31543 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.693406 31543 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.695536 31243 raft_consensus.cc:348] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.695976 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.696334 31243 raft_consensus.cc:374] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.696707 31243 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.696892 31054 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:31.697613 31243 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.700060 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.702975 31177 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:31.703246 31557 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:31.711899 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.711990 31558 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:31.712908 30904 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:31.714145 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:31.718504 31554 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.729872 31243 ts_tablet_manager.cc:1428] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260809 07:32:31.732741 31243 tablet_bootstrap.cc:492] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:31.742274 31554 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.764750 31243 tablet_bootstrap.cc:654] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.761734 31543 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.771651 31245 tablet_bootstrap.cc:492] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.772104 31245 ts_tablet_manager.cc:1397] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.137s user 0.008s sys 0.017s
I20260809 07:32:31.776969 31245 raft_consensus.cc:348] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.777745 31245 raft_consensus.cc:374] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.778026 31245 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.778378 31244 tablet_bootstrap.cc:492] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.778791 31244 ts_tablet_manager.cc:1397] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.130s user 0.016s sys 0.006s
I20260809 07:32:31.778638 31245 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.783836 31543 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.783978 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:31.784618 31055 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.789309 31245 ts_tablet_manager.cc:1428] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260809 07:32:31.788842 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:31.794741 31245 tablet_bootstrap.cc:492] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:31.794620 31177 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.800091 31244 raft_consensus.cc:348] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.800802 31244 raft_consensus.cc:374] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.807894 31244 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.808290 31543 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.808835 31543 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.811280 31543 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.813717 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.815357 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.810509 31244 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.816134 31055 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
W20260809 07:32:31.815387 30853 leader_election.cc:341] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:31.822376 30852 leader_election.cc:341] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.822885 31556 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.823446 31556 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.824771 30852 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:31.825660 31556 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:31.826002 31545 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.826670 31545 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:31.829820 30852 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:31.830616 31545 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:31.831166 31545 raft_consensus.cc:686] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:31.831801 31556 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:31.831928 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:31.832094 31545 consensus_queue.cc:237] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.835803 31244 ts_tablet_manager.cc:1428] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.057s user 0.004s sys 0.002s
I20260809 07:32:31.836767 31244 tablet_bootstrap.cc:492] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:31.842808 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:31.844580 31120 consensus_peers.cc:487] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 -> Peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Couldn't send request to peer 2c9de56c3fa3487c9dac990675c34be5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:31.847343 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:31.848028 31556 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:31.848989 31245 tablet_bootstrap.cc:654] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.849856 31177 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:31.854903 31243 tablet_bootstrap.cc:492] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:31.855381 31243 ts_tablet_manager.cc:1397] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.123s user 0.019s sys 0.016s
I20260809 07:32:31.870710 31244 tablet_bootstrap.cc:654] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.871142 30744 catalog_manager.cc:5495] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:31.875461 31243 raft_consensus.cc:348] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.876185 31243 raft_consensus.cc:374] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.876477 31243 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.880677 31181 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.888093 31243 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.891908 31243 ts_tablet_manager.cc:1428] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.036s user 0.008s sys 0.000s
I20260809 07:32:31.893551 31243 tablet_bootstrap.cc:492] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:31.895433 31245 tablet_bootstrap.cc:492] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:31.898783 31554 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:31.906596 31245 ts_tablet_manager.cc:1397] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.112s user 0.006s sys 0.011s
I20260809 07:32:31.899514 31177 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.909838 31245 raft_consensus.cc:348] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.910506 31245 raft_consensus.cc:374] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.910800 31245 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.917094 31245 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.927970 31545 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.928610 31545 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.940093 31543 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:31.943974 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:31.944736 31051 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:31.945815 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:31.946605 31545 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.947168 31545 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:31.947510 31545 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:31.947952 31545 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:31.949038 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.951337 31245 ts_tablet_manager.cc:1428] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
W20260809 07:32:31.951508 30853 leader_election.cc:341] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:31.952133 31244 tablet_bootstrap.cc:492] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:31.952553 31244 ts_tablet_manager.cc:1397] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.116s user 0.006s sys 0.012s
I20260809 07:32:31.953660 31245 tablet_bootstrap.cc:492] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:31.961227 31244 raft_consensus.cc:348] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.961941 31244 raft_consensus.cc:374] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:31.962193 31244 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:31.962842 31244 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.967550 31244 ts_tablet_manager.cc:1428] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.015s user 0.004s sys 0.006s
I20260809 07:32:31.968467 31244 tablet_bootstrap.cc:492] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:31.974217 31243 tablet_bootstrap.cc:654] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:31.983381 31557 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:31.984139 31557 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:31.991201 31557 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:31.996107 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:31.996966 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:32.015358 30906 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:32.007884 31006 leader_election.cc:341] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:32.018457 31005 leader_election.cc:341] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.019049 31005 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:32.019980 31557 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:32.021484 31244 tablet_bootstrap.cc:654] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.041391 31245 tablet_bootstrap.cc:654] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.038246 31545 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.075815 30904 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.076776 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:32.077498 31055 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.079733 31557 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.080281 31557 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.084950 31545 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.086117 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:32.087970 31557 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:32.093400 30853 leader_election.cc:341] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.096151 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:32.096330 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:32.096900 30911 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:32.097028 31181 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:32.098654 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:32.099608 31557 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:32.099993 31557 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:32.100370 31557 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:32.109896 31243 tablet_bootstrap.cc:492] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:32.110359 31243 ts_tablet_manager.cc:1397] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.217s user 0.007s sys 0.011s
I20260809 07:32:32.117848 31245 tablet_bootstrap.cc:492] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:32.118343 31245 ts_tablet_manager.cc:1397] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.165s user 0.003s sys 0.025s
I20260809 07:32:32.119123 31243 raft_consensus.cc:348] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.119601 31243 raft_consensus.cc:374] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.119801 31243 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.122493 31666 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:32.122632 31557 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.123306 31245 raft_consensus.cc:348] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.123994 31245 raft_consensus.cc:374] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.124261 31245 raft_consensus.cc:729] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.124758 31545 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.125288 31545 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.126644 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:32.127228 31681 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:32.127416 31243 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.129274 31245 consensus_queue.cc:260] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.129997 31243 ts_tablet_manager.cc:1428] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260809 07:32:32.121208 31055 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:32.130877 31243 tablet_bootstrap.cc:492] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:32.127310 31181 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:32.132051 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:32.132709 30909 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:32.134445 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:32.139930 31245 ts_tablet_manager.cc:1428] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260809 07:32:32.139822 31680 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:32.140504 31680 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:32.141512 31680 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.143729 30909 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:32.145138 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:32.146126 31558 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:32.146663 31558 raft_consensus.cc:686] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:32.147504 31558 consensus_queue.cc:237] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.150998 31245 tablet_bootstrap.cc:492] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:32.161234 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:32.163297 30852 leader_election.cc:341] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.167342 31667 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.154938 31557 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.172818 31667 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.174965 31545 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.177407 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:32.178402 31177 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:32.179613 31667 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:32.180117 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:32.181351 31181 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:32.182036 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:32.182792 31543 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:32.183349 31055 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:32.186901 31244 tablet_bootstrap.cc:492] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:32.187297 31244 ts_tablet_manager.cc:1397] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.219s user 0.020s sys 0.042s
I20260809 07:32:32.183422 31543 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:32.191516 31543 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.198873 31631 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.199548 31631 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.199904 30744 catalog_manager.cc:5495] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 2, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:32.202702 31244 raft_consensus.cc:348] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.191871 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:32.203509 31244 raft_consensus.cc:374] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.204324 31244 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.204788 31244 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.208271 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:32.212520 30899 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:32.214541 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:32.215600 31631 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.215831 31490 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:32.218204 31490 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:32.218631 31490 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.217077 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:32.223064 31006 leader_election.cc:341] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.227867 31490 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.233511 31490 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.238446 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:32.239951 31243 tablet_bootstrap.cc:654] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:32.240242 31006 leader_election.cc:341] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.238869 31244 ts_tablet_manager.cc:1428] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.051s user 0.005s sys 0.004s
I20260809 07:32:32.241251 30745 catalog_manager.cc:5495] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:32.242556 31244 tablet_bootstrap.cc:492] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:32.244894 31245 tablet_bootstrap.cc:654] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.248703 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:32.249907 31681 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.254381 31554 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:32.241472 31543 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.257620 31543 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.272526 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:32.277066 31681 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:32:32.282858 30852 leader_election.cc:341] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.286444 31244 tablet_bootstrap.cc:654] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.293119 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:32.293511 31245 tablet_bootstrap.cc:492] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:32.293730 30904 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.293928 31245 ts_tablet_manager.cc:1397] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.143s user 0.004s sys 0.021s
I20260809 07:32:32.346131 31243 tablet_bootstrap.cc:492] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:32.346662 31243 ts_tablet_manager.cc:1397] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.216s user 0.016s sys 0.071s
I20260809 07:32:32.347886 31245 raft_consensus.cc:348] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.348625 31245 raft_consensus.cc:374] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.348896 31245 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.349089 31243 raft_consensus.cc:348] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.349857 31243 raft_consensus.cc:374] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.307269 30904 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:32.350279 31243 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.351195 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:32.351257 31243 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.352388 31558 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:32.352851 31558 raft_consensus.cc:686] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:32.353240 31243 ts_tablet_manager.cc:1428] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:32:32.353679 31558 consensus_queue.cc:237] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.356937 31243 tablet_bootstrap.cc:492] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:32.313113 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:32.364274 31181 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.367062 31243 tablet_bootstrap.cc:654] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.370440 31244 tablet_bootstrap.cc:492] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:32.370852 31244 ts_tablet_manager.cc:1397] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.130s user 0.033s sys 0.011s
I20260809 07:32:32.372550 30904 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.373211 31244 raft_consensus.cc:348] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.373893 31244 raft_consensus.cc:374] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.332558 31490 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.376199 31490 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.378952 31490 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.378861 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:32.380029 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:32.380713 31005 leader_election.cc:341] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:32.381861 31006 leader_election.cc:341] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.382267 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:32.383302 31631 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:32.374148 31244 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.387777 31244 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.390066 31244 ts_tablet_manager.cc:1428] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260809 07:32:32.391002 31244 tablet_bootstrap.cc:492] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:32.394475 31245 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.399659 31245 ts_tablet_manager.cc:1428] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.055s user 0.005s sys 0.001s
I20260809 07:32:32.400545 31245 tablet_bootstrap.cc:492] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:32.403018 31243 tablet_bootstrap.cc:492] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:32.403537 31243 ts_tablet_manager.cc:1397] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.003s
I20260809 07:32:32.410013 31244 tablet_bootstrap.cc:654] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.417670 31181 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:32.423408 30745 catalog_manager.cc:5495] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:32.436226 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:32.440267 31542 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:32.440929 31542 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:32.442566 31542 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.447881 31243 raft_consensus.cc:348] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.448751 31243 raft_consensus.cc:374] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.449018 31243 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.449566 31243 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.453655 31245 tablet_bootstrap.cc:654] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.454742 31692 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:32.456643 31243 ts_tablet_manager.cc:1428] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.053s user 0.003s sys 0.007s
I20260809 07:32:32.463640 31667 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.464273 31667 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.466125 31692 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:32.471508 31667 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:32.471635 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:32.472595 30911 raft_consensus.cc:2379] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:32.479255 31243 tablet_bootstrap.cc:492] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:32.482983 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:32.488376 31120 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:32.489496 31681 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.496632 30742 catalog_manager.cc:5495] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:32.507493 31681 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:32.547977 31243 tablet_bootstrap.cc:654] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.555625 31244 tablet_bootstrap.cc:492] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:32.556223 31244 ts_tablet_manager.cc:1397] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.165s user 0.017s sys 0.017s
I20260809 07:32:32.569648 31245 tablet_bootstrap.cc:492] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:32.570289 31245 ts_tablet_manager.cc:1397] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.170s user 0.024s sys 0.024s
I20260809 07:32:32.567783 31244 raft_consensus.cc:348] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.573768 31245 raft_consensus.cc:348] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.574373 31244 raft_consensus.cc:374] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.574551 31245 raft_consensus.cc:374] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.574806 31244 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.574888 31245 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.575238 31558 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.576112 31244 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.576503 31558 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.580219 31245 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.581317 31244 ts_tablet_manager.cc:1428] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.023s user 0.008s sys 0.000s
I20260809 07:32:32.582947 31244 tablet_bootstrap.cc:492] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:32.583630 31490 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.584357 31490 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.588423 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:32.591202 31005 leader_election.cc:341] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.588502 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:32.593824 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:32.597509 31005 leader_election.cc:341] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.585886 31558 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.598789 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:32.601047 31006 leader_election.cc:341] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.601437 31006 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:32.590184 31490 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.603556 31490 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:32:32.604313 31006 leader_election.cc:341] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.604815 31006 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:32.605857 31558 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:32.608477 31245 ts_tablet_manager.cc:1428] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.038s user 0.006s sys 0.002s
I20260809 07:32:32.609395 31245 tablet_bootstrap.cc:492] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:32.618865 31244 tablet_bootstrap.cc:654] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.636987 31243 tablet_bootstrap.cc:492] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:32.637460 31243 ts_tablet_manager.cc:1397] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.158s user 0.027s sys 0.025s
I20260809 07:32:32.642997 31245 tablet_bootstrap.cc:654] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.643458 31243 raft_consensus.cc:348] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.644176 31243 raft_consensus.cc:374] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.644469 31243 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.645224 31243 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.653234 31243 ts_tablet_manager.cc:1428] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.015s user 0.001s sys 0.009s
I20260809 07:32:32.655181 31243 tablet_bootstrap.cc:492] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:32.658102 31545 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:32.664762 31545 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.665419 31545 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.674829 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:32.674258 31545 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.675554 31054 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:32.677009 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:32.677310 30911 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260809 07:32:32.677779 31545 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:32.678351 31545 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:32.678823 31545 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.679090 31558 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:32.677373 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:32.681257 30853 leader_election.cc:341] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.689109 31243 tablet_bootstrap.cc:654] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.690577 31244 tablet_bootstrap.cc:492] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:32.691013 31244 ts_tablet_manager.cc:1397] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.108s user 0.013s sys 0.005s
I20260809 07:32:32.722908 31244 raft_consensus.cc:348] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.723690 31244 raft_consensus.cc:374] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.723973 31244 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.724547 31244 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.725957 31245 tablet_bootstrap.cc:492] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:32.711593 31692 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.726430 31245 ts_tablet_manager.cc:1397] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.117s user 0.027s sys 0.008s
I20260809 07:32:32.727998 31244 ts_tablet_manager.cc:1428] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.037s user 0.008s sys 0.000s
I20260809 07:32:32.728729 31244 tablet_bootstrap.cc:492] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:32.726531 31692 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.742514 31692 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.781437 31244 tablet_bootstrap.cc:654] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:32.843979 31245 raft_consensus.cc:348] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.844750 31245 raft_consensus.cc:374] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.845014 31245 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.845604 31245 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.847718 31245 ts_tablet_manager.cc:1428] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.121s user 0.023s sys 0.014s
W20260809 07:32:32.858366 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 77225.2us
I20260809 07:32:32.757418 31175 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260809 07:32:32.744266 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:32.862280 31243 tablet_bootstrap.cc:492] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:32.862890 31243 ts_tablet_manager.cc:1397] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.208s user 0.010s sys 0.024s
I20260809 07:32:32.753505 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:32.869239 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:32.871824 31055 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:32.878973 31243 raft_consensus.cc:348] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.879735 31243 raft_consensus.cc:374] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:32.880007 31243 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:32.880672 31243 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.810711 31545 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.892336 31243 ts_tablet_manager.cc:1428] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260809 07:32:32.896834 31243 tablet_bootstrap.cc:492] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:32.884171 31558 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.906472 31245 tablet_bootstrap.cc:492] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:32.911044 30852 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:32.895304 31490 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260809 07:32:32.912341 31680 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:32.912707 31680 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:32.913110 31680 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:32.931905 30853 leader_election.cc:341] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:32.935067 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:32.935920 31051 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:32.942692 31725 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.943300 31725 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.910913 31558 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.946187 31725 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:32.966442 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 50981.6us
I20260809 07:32:32.972653 31542 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.980908 31681 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:32.981554 31681 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.984637 31681 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:32.973279 31542 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:32.973269 31545 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.032270 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:33.034248 31558 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:32.998999 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:32.995625 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 60545.4us
I20260809 07:32:33.069053 31243 tablet_bootstrap.cc:654] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:33.000854 31680 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.061911 31692 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260809 07:32:33.078166 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:33.078822 31058 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:33.078984 31244 tablet_bootstrap.cc:492] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:33.080235 31244 ts_tablet_manager.cc:1397] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.352s user 0.018s sys 0.026s
I20260809 07:32:33.084646 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.089147 31681 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:33.090500 31681 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:33.090845 31681 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.090688 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:33.093889 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:33.097608 31542 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.090497 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:33.099258 31120 leader_election.cc:341] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.101527 31680 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.108557 31244 raft_consensus.cc:348] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.108809 31245 tablet_bootstrap.cc:654] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:33.109927 31120 leader_election.cc:341] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:33.113531 30853 leader_election.cc:341] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.110997 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:33.121785 31244 raft_consensus.cc:374] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 07:32:33.185700 31734 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
I20260809 07:32:33.193261 31244 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.129630 31730 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260809 07:32:33.188877 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:33.188900 31681 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:33.188992 31005 leader_election.cc:341] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.190748 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:33.201201 31057 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.202632 31724 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:33.203334 31724 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.191551 31051 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:33.193988 31244 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:33.195207 30852 consensus_peers.cc:487] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 -> Peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Couldn't send request to peer a720817fc113446a8ed55e61523b280a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 07:32:33.202695 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 75142.6us
W20260809 07:32:33.207307 31120 leader_election.cc:341] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.207916 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:33.212733 31049 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:33.213855 31666 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:33.215605 31724 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:33.221477 31006 leader_election.cc:341] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.221923 31006 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:33.228269 31738 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:33.228432 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:33.230285 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:33.232044 31244 ts_tablet_manager.cc:1428] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.152s user 0.009s sys 0.005s
I20260809 07:32:33.233042 31244 tablet_bootstrap.cc:492] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:33.234565 31055 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:33.239701 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:33.240587 30911 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:33.243150 31545 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:33.243741 31545 raft_consensus.cc:686] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:33.244577 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.245283 31725 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:33.245626 31681 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:33.245805 31725 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:33.246477 31725 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.244939 31545 consensus_queue.cc:237] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.246636 31243 tablet_bootstrap.cc:492] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:33.249150 31243 ts_tablet_manager.cc:1397] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.352s user 0.039s sys 0.056s
I20260809 07:32:33.244971 30852 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.255246 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:33.253644 31543 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:33.256214 31051 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.256533 31543 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:33.256911 31057 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:33.257150 31543 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.260769 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:33.262296 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 53898us
W20260809 07:32:33.266387 31120 leader_election.cc:341] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.269410 30852 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.299544 31730 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260809 07:32:33.319140 31243 raft_consensus.cc:348] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:33.320405 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 45094.3us
I20260809 07:32:33.319872 31243 raft_consensus.cc:374] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.323197 31243 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.325562 31245 tablet_bootstrap.cc:492] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:33.327536 31245 ts_tablet_manager.cc:1397] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.421s user 0.014s sys 0.012s
I20260809 07:32:33.328814 31243 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.341665 31244 tablet_bootstrap.cc:654] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:33.346655 31243 ts_tablet_manager.cc:1428] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.097s user 0.005s sys 0.007s
I20260809 07:32:33.347920 31243 tablet_bootstrap.cc:492] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:33.357666 31245 raft_consensus.cc:348] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.358434 31245 raft_consensus.cc:374] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.358702 31245 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.360548 31245 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.364238 31746 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:33.382270 31725 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.390754 31746 raft_consensus.cc:686] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:33.392294 31746 consensus_queue.cc:237] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.395252 31245 ts_tablet_manager.cc:1428] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.067s user 0.003s sys 0.004s
I20260809 07:32:33.396100 31245 tablet_bootstrap.cc:492] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:33.391011 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:33.407961 31051 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:33.409883 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:33.423269 30744 catalog_manager.cc:5495] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:32:33.454082 31757 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.081s user 0.001s sys 0.000s
I20260809 07:32:33.420974 31666 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:33.441687 31724 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:33.447878 31542 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:33.453778 31543 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.491837 31725 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:33.469591 31666 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:33.496003 31245 tablet_bootstrap.cc:654] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:33.480291 31542 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.497294 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:33.504969 31744 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 07:32:33.500545 31542 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.478956 31724 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.518080 31666 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:33.547935 30853 leader_election.cc:341] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.541489 31543 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.554940 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:33.552479 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:33.555631 31049 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.555747 31055 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:33.558069 31724 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:33.547102 31172 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:33.574709 31243 tablet_bootstrap.cc:654] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:33.574900 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:33.575546 31054 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.577060 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:33.577646 30906 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.576382 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:33.590437 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.592197 31746 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:33.594764 31746 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:33.595098 31746 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.606402 31244 tablet_bootstrap.cc:492] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:33.606889 31244 ts_tablet_manager.cc:1397] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.374s user 0.034s sys 0.029s
I20260809 07:32:33.607846 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:33.608669 30899 raft_consensus.cc:2379] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:33.612717 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:33.616201 31054 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:33.617057 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:33.629328 31245 tablet_bootstrap.cc:492] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:33.629956 31245 ts_tablet_manager.cc:1397] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.234s user 0.005s sys 0.021s
I20260809 07:32:33.631817 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:33.634274 30852 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.641492 31680 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:33.642068 31680 raft_consensus.cc:686] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:33.642964 31680 consensus_queue.cc:237] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:33.645339 30853 leader_election.cc:341] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.636986 31667 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.655500 31746 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.657670 31746 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.660190 31244 raft_consensus.cc:348] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.661024 31244 raft_consensus.cc:374] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.661288 31244 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.661947 31244 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.668659 31049 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:33.691572 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:33.693220 31681 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:33.693730 31681 raft_consensus.cc:686] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:33.694947 30906 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:33.696890 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:33.699877 30853 leader_election.cc:341] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.701268 31244 ts_tablet_manager.cc:1428] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.094s user 0.007s sys 0.000s
I20260809 07:32:33.703452 31244 tablet_bootstrap.cc:492] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:33.699391 31681 consensus_queue.cc:237] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.719012 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:33.719655 31051 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.720336 31667 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:33.725165 31243 tablet_bootstrap.cc:492] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:33.726006 31243 ts_tablet_manager.cc:1397] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.378s user 0.024s sys 0.029s
I20260809 07:32:33.726977 31245 raft_consensus.cc:348] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.727861 31245 raft_consensus.cc:374] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.728161 31245 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.728858 31245 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.731611 31245 ts_tablet_manager.cc:1428] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.101s user 0.006s sys 0.000s
I20260809 07:32:33.732776 31245 tablet_bootstrap.cc:492] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:33.733682 31243 raft_consensus.cc:348] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.744835 31243 raft_consensus.cc:374] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.745118 31243 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.752534 31051 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:33.754418 31243 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.757666 31244 tablet_bootstrap.cc:654] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:33.759958 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:33.760864 31680 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:33.761570 31680 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:33.762843 31680 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.766364 31243 ts_tablet_manager.cc:1428] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.040s user 0.008s sys 0.002s
I20260809 07:32:33.767063 31243 tablet_bootstrap.cc:492] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:33.765765 30745 catalog_manager.cc:5495] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:33.803221 31245 tablet_bootstrap.cc:654] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:33.804034 30853 consensus_peers.cc:487] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:33.809489 31745 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:33.810048 31745 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.813786 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:33.823865 31243 tablet_bootstrap.cc:654] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:33.846529 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:33.858927 31006 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:33.859769 31557 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:33.860455 30744 catalog_manager.cc:5495] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:33.848011 31745 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:33.862660 30744 catalog_manager.cc:5495] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:33.866174 31165 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:33.864694 30744 catalog_manager.cc:5495] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:33.886976 31744 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:33.887140 31680 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:33.908936 31244 tablet_bootstrap.cc:492] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:33.909439 31244 ts_tablet_manager.cc:1397] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.206s user 0.010s sys 0.019s
I20260809 07:32:33.921165 31244 raft_consensus.cc:348] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.930140 31244 raft_consensus.cc:374] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.930464 31244 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.908512 30742 catalog_manager.cc:5495] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:33.931139 31244 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.933346 31244 ts_tablet_manager.cc:1428] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.024s user 0.003s sys 0.004s
I20260809 07:32:33.934182 31244 tablet_bootstrap.cc:492] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:33.943769 31245 tablet_bootstrap.cc:492] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:33.954219 31557 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:33.954778 31680 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 07:32:33.957926 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:33.958544 31557 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.974457 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:33.975322 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:33.977392 31005 leader_election.cc:341] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:33.979687 31006 leader_election.cc:341] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:33.980105 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:33.980943 31738 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:33.983666 31245 ts_tablet_manager.cc:1397] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.251s user 0.007s sys 0.025s
I20260809 07:32:33.993579 31245 raft_consensus.cc:348] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.994493 31245 raft_consensus.cc:374] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:33.994807 31245 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:33.995386 31245 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:33.999603 31245 ts_tablet_manager.cc:1428] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260809 07:32:34.000552 31245 tablet_bootstrap.cc:492] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:33.987206 31557 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:34.023175 31243 tablet_bootstrap.cc:492] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:34.023936 31243 ts_tablet_manager.cc:1397] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.257s user 0.015s sys 0.028s
I20260809 07:32:34.030961 31244 tablet_bootstrap.cc:654] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:34.038259 31243 raft_consensus.cc:348] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.054816 31243 raft_consensus.cc:374] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:34.055060 31243 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:34.060156 31243 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.068616 31245 tablet_bootstrap.cc:654] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:34.070911 31744 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:34.071633 31744 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.073344 31055 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:34.075166 31243 ts_tablet_manager.cc:1428] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.051s user 0.003s sys 0.003s
I20260809 07:32:34.076081 31243 tablet_bootstrap.cc:492] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:34.081645 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:34.083585 30852 leader_election.cc:341] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:34.084504 31744 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:34.099613 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:34.100350 31177 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:34.101485 30853 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:34.103695 31744 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:34.115406 31744 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:34.115944 31744 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:34.136555 31244 tablet_bootstrap.cc:492] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:34.137033 31244 ts_tablet_manager.cc:1397] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.203s user 0.016s sys 0.028s
W20260809 07:32:34.140961 31120 consensus_peers.cc:487] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 -> Peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Couldn't send request to peer 2c9de56c3fa3487c9dac990675c34be5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:34.140681 31244 raft_consensus.cc:348] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.141559 31244 raft_consensus.cc:374] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:34.141835 31244 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:34.143723 31244 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.148450 31244 ts_tablet_manager.cc:1428] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.011s user 0.004s sys 0.003s
I20260809 07:32:34.149295 31244 tablet_bootstrap.cc:492] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:34.154230 31245 tablet_bootstrap.cc:492] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:34.154826 31245 ts_tablet_manager.cc:1397] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.154s user 0.009s sys 0.008s
I20260809 07:32:34.160071 31243 tablet_bootstrap.cc:654] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:34.154029 31726 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 07:32:34.162137 31667 consensus_queue.cc:1035] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:34.172847 31667 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:34.173003 31245 raft_consensus.cc:348] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.173779 31245 raft_consensus.cc:374] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:34.173326 31667 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.174054 31245 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:34.176304 31667 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:34.177839 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:34.179345 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:34.180743 31057 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
W20260809 07:32:34.181887 31120 leader_election.cc:341] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:34.184501 31744 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.187376 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:34.177168 31245 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.214701 31244 tablet_bootstrap.cc:654] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:34.212867 31245 ts_tablet_manager.cc:1428] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.058s user 0.001s sys 0.006s
I20260809 07:32:34.220582 31245 tablet_bootstrap.cc:492] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:34.226902 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:34.227677 31054 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:34.230087 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:34.230845 31165 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:34.217716 31744 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:34.198113 31667 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:34.236143 31667 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:34.259275 31667 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:34.256878 31543 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260809 07:32:34.275832 31245 tablet_bootstrap.cc:654] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:34.275213 31543 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 07:32:34.280457 31744 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:34.328219 31054 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:34.294672 31744 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.344719 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:34.346763 31746 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:34.347399 31746 raft_consensus.cc:686] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:34.348588 31746 consensus_queue.cc:237] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.365073 31244 tablet_bootstrap.cc:492] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:34.365630 31244 ts_tablet_manager.cc:1397] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.217s user 0.008s sys 0.013s
I20260809 07:32:34.368531 31244 raft_consensus.cc:348] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.369239 31244 raft_consensus.cc:374] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:34.369482 31244 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:34.370079 31244 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.377527 31243 tablet_bootstrap.cc:492] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:34.378120 31243 ts_tablet_manager.cc:1397] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.302s user 0.015s sys 0.025s
W20260809 07:32:34.378826 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 84124.3us
I20260809 07:32:34.384613 31244 ts_tablet_manager.cc:1428] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.019s user 0.004s sys 0.004s
I20260809 07:32:34.391049 31165 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:34.393389 31744 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:34.397576 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:34.398406 31811 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.102s user 0.000s sys 0.005s
I20260809 07:32:34.400063 31817 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:34.401604 31817 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:34.410934 30853 leader_election.cc:341] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:34.460490 31244 tablet_bootstrap.cc:492] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
W20260809 07:32:34.493263 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 175629us
I20260809 07:32:34.588294 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:34.589071 31817 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:34.589327 31051 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
W20260809 07:32:34.704356 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 201280us
I20260809 07:32:34.543185 31667 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.731189 31818 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:34.803995 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 296204us
I20260809 07:32:34.870376 31245 tablet_bootstrap.cc:492] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:34.872717 31245 ts_tablet_manager.cc:1397] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.652s user 0.026s sys 0.034s
I20260809 07:32:34.834661 31667 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:34.873211 31243 raft_consensus.cc:348] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.907930 31243 raft_consensus.cc:374] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:34.908003 31244 tablet_bootstrap.cc:654] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:34.908550 31243 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:34.909299 31243 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.756791 31818 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.935838 31818 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:34.948724 31245 raft_consensus.cc:348] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:34.949589 31245 raft_consensus.cc:374] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:34.949851 31245 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:34.951279 31245 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:34.972117 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 97567.9us
I20260809 07:32:34.978929 31245 ts_tablet_manager.cc:1428] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.106s user 0.006s sys 0.000s
I20260809 07:32:34.979995 31245 tablet_bootstrap.cc:492] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:35.000149 31244 tablet_bootstrap.cc:492] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:35.000710 31244 ts_tablet_manager.cc:1397] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.540s user 0.015s sys 0.014s
W20260809 07:32:35.015800 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260809 07:32:35.018918 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 56109.8us
I20260809 07:32:35.027318 31243 ts_tablet_manager.cc:1428] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.649s user 0.020s sys 0.036s
W20260809 07:32:35.064504 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 44798.4us
I20260809 07:32:35.066226 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:35.019470 31745 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:35.027715 31243 ts_tablet_manager.cc:1460] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Tablet startup took 649ms
I20260809 07:32:35.080720 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:35.081473 31051 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:35.085606 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 82858.3us
I20260809 07:32:35.026288 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:35.090988 31244 raft_consensus.cc:348] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.091661 31244 raft_consensus.cc:374] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:35.091921 31244 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:35.094025 31244 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.096872 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:35.104875 31244 ts_tablet_manager.cc:1428] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.104s user 0.007s sys 0.000s
I20260809 07:32:35.090179 31817 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:35.074645 31680 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260809 07:32:35.109787 31244 tablet_bootstrap.cc:492] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
W20260809 07:32:35.075278 31243 ts_tablet_manager.cc:1462] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Trace:
0809 07:32:26.530502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:26.885165 (+354663us) service_pool.cc:227] Handling call
0809 07:32:26.914385 (+ 29220us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 07:32:26.914476 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0809 07:32:27.004418 (+ 89942us) tablet_metadata.cc:641] Metadata flushed
0809 07:32:27.082654 (+ 78236us) tablet_replica.cc:205] Creating consensus instance
0809 07:32:27.083099 (+ 445us) inbound_call.cc:173] Queueing success response
0809 07:32:34.075779 (+6992680us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 07:32:34.378457 (+302678us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 07:32:34.592936 (+214479us) tablet_replica.cc:259] Starting instrumentation
0809 07:32:34.593490 (+ 554us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":34972,"raft.queue_time_us":71472,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":4736,"tablet-open.queue_time_us":6992744,"thread_start_us":99430,"threads_started":1}
I20260809 07:32:35.111047 31243 tablet_bootstrap.cc:492] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:35.116562 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:35.085762 31745 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.108642 31817 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:35.118588 31817 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:35.108990 31005 leader_election.cc:341] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.122726 31245 tablet_bootstrap.cc:654] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:35.187078 31120 leader_election.cc:341] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.113193 31824 consensus_queue.cc:1035] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.074s
I20260809 07:32:35.153496 31046 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:35.198704 30898 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.200500 31243 tablet_bootstrap.cc:654] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:35.206045 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:35.215072 31006 leader_election.cc:341] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.215489 31006 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:35.216650 30853 consensus_peers.cc:487] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:35.217046 31860 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:35.218297 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:35.219087 30906 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 0.
I20260809 07:32:35.219801 31051 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:35.225083 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:35.225447 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:35.226363 31745 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:35.227414 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:35.232126 31840 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:35.232885 31840 raft_consensus.cc:686] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:35.239956 31745 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:35.244258 31745 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:35.244797 31745 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.238669 31840 consensus_queue.cc:237] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.239207 31848 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260809 07:32:35.330783 31823 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.734s user 0.016s sys 0.008s
I20260809 07:32:35.306771 31726 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:35.344336 31726 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.369652 31726 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:35.429661 31861 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.192s user 0.000s sys 0.001s
I20260809 07:32:35.434682 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 168) took 1153 ms. Trace:
I20260809 07:32:35.435009 31726 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:35.442444 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:35.461082 31244 tablet_bootstrap.cc:654] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:35.432116 31745 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.456285 31817 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.441417 31059 rpcz_store.cc:276] 0809 07:32:34.285342 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:34.290195 (+ 4853us) service_pool.cc:227] Handling call
0809 07:32:34.290707 (+ 512us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:34.293123 (+ 2416us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:34.293158 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:34.293834 (+ 676us) log.cc:842] Serialized 26 byte log entry
0809 07:32:34.383016 (+ 89182us) raft_consensus.cc:1631] Marking committed up to 0
0809 07:32:34.384659 (+ 1643us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:34.385850 (+ 1191us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:35.244776 (+858926us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:35.434170 (+189394us) raft_consensus.cc:1671] finished
0809 07:32:35.434191 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:35.434562 (+ 371us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":670,"raft.queue_time_us":3240,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":15646,"spinlock_wait_cycles":68935040,"thread_start_us":80682,"threads_started":1,"wal-append.queue_time_us":125555}
I20260809 07:32:35.471830 31726 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.507433 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:35.508345 30906 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.515364 31726 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:35.522629 31745 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:35.523300 31745 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:35.523757 31745 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:35.518335 30853 leader_election.cc:341] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:35.525384 31006 leader_election.cc:341] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.530494 31245 tablet_bootstrap.cc:492] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:35.530942 31245 ts_tablet_manager.cc:1397] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.551s user 0.014s sys 0.060s
I20260809 07:32:35.512773 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:35.537696 31746 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:35.534852 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:35.544898 31835 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:35.545352 31835 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.547266 31835 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:35.546165 31745 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:35.535089 31817 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:35.554153 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:35.552002 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:35.555864 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:35.556036 31245 raft_consensus.cc:348] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.556773 31245 raft_consensus.cc:374] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:35.557041 31245 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:35.557232 31165 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.557608 31245 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.557941 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:35.562786 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:35.562871 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:35.556615 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:35.564280 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:35.567348 31005 leader_election.cc:341] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:35.567961 31006 leader_election.cc:341] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.568645 31006 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:35.569489 31726 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:35.569985 31726 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.572124 31726 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:35.573866 30853 leader_election.cc:341] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.575855 31835 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:35.576550 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:35.578795 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:35.583151 31245 ts_tablet_manager.cc:1428] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.052s user 0.007s sys 0.000s
I20260809 07:32:35.582433 30745 catalog_manager.cc:5495] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:35.590221 31245 tablet_bootstrap.cc:492] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:35.592437 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40566 (request call id 111) took 1075 ms. Trace:
I20260809 07:32:35.594880 30742 catalog_manager.cc:5495] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:35.598104 31244 tablet_bootstrap.cc:492] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:35.598631 31244 ts_tablet_manager.cc:1397] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.493s user 0.010s sys 0.013s
W20260809 07:32:35.599740 31006 leader_election.cc:341] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.600387 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:35.607620 31243 tablet_bootstrap.cc:492] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:35.608091 31243 ts_tablet_manager.cc:1397] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.497s user 0.032s sys 0.034s
I20260809 07:32:35.594841 30744 rpcz_store.cc:276] 0809 07:32:34.507131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:34.507270 (+ 139us) service_pool.cc:227] Handling call
0809 07:32:35.582440 (+1075170us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9968,"reported_tablets":3}
I20260809 07:32:35.582232 31745 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:35.615865 31244 raft_consensus.cc:348] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.616603 31244 raft_consensus.cc:374] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:35.616864 31244 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:35.615456 31243 raft_consensus.cc:348] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.618161 31243 raft_consensus.cc:374] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:35.618759 31243 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:35.619396 31244 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.628022 31243 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.621308 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:35.630271 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:35.632123 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:35.614753 31840 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:35.636368 31244 ts_tablet_manager.cc:1428] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.037s user 0.004s sys 0.003s
I20260809 07:32:35.637305 31244 tablet_bootstrap.cc:492] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:35.647262 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:35.648839 31243 ts_tablet_manager.cc:1428] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260809 07:32:35.650484 31243 tablet_bootstrap.cc:492] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:35.655697 31681 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:35.655936 31847 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:35.659689 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:35.660274 31049 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.674413 30906 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:35.675132 31680 raft_consensus.cc:2741] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 07:32:35.680542 31744 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:35.681000 31744 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.699885 31049 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:35.711828 31744 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:35.711894 31181 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.713500 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:35.714154 31245 tablet_bootstrap.cc:654] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:35.714932 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:35.725914 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:35.726771 31820 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:35.727242 31820 raft_consensus.cc:686] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:35.715477 31051 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:35.719022 31680 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:35.739535 31680 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:35.740845 31680 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.743280 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:35.739013 30852 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:35.754422 31744 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 07:32:35.755051 30853 leader_election.cc:341] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:35.755043 31744 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:35.755748 31744 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:35.728083 31820 consensus_queue.cc:237] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:35.811645 31244 tablet_bootstrap.cc:654] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:35.812680 31243 tablet_bootstrap.cc:654] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:35.806231 31744 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:35.958112 31120 consensus_peers.cc:487] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 -> Peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Couldn't send request to peer 2c9de56c3fa3487c9dac990675c34be5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:35.968940 31744 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.009995 31245 tablet_bootstrap.cc:492] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:36.011121 31245 ts_tablet_manager.cc:1397] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.427s user 0.023s sys 0.005s
I20260809 07:32:36.010967 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:36.021791 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40564 (request call id 112) took 1514 ms. Trace:
I20260809 07:32:36.023473 31745 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.024405 31745 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.044715 31725 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260809 07:32:36.022294 30745 rpcz_store.cc:276] 0809 07:32:34.677200 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:34.691876 (+ 14676us) service_pool.cc:227] Handling call
0809 07:32:35.010726 (+318850us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b5400002728. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409b282501 00007f409b27d5bc 00007f409ae53d54 00007f409af4dc3e 00007f409925ffb4 00007f409925ff42 00007f409925fe71 00007f409925fdec 00007f409925f0a2 00007f409332394c 00007f4093322d86
0809 07:32:35.227851 (+217125us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5400002dd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7eaf0 00007f4094f01f2a 00007f409ae53d6c 00007f409af4dc3e 00007f409925ffb4 00007f409925ff42 00007f409925fe71 00007f409925fdec 00007f409925f0a2 00007f409332394c 00007f4093322d86
0809 07:32:36.021556 (+793705us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:32:35.602758 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:32:35.603311 (+ 553us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:32:35.603342 (+ 31us) write_op.cc:435] Acquired schema lock
0809 07:32:35.603359 (+ 17us) tablet.cc:660] Decoding operations
0809 07:32:35.603680 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:32:35.603758 (+ 78us) write_op.cc:641] Partition lock acquired for write op
0809 07:32:35.603802 (+ 44us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:32:35.604012 (+ 210us) tablet.cc:699] Row locks acquired
0809 07:32:35.604031 (+ 19us) write_op.cc:260] PREPARE: finished
0809 07:32:35.604169 (+ 138us) write_op.cc:270] Start()
0809 07:32:35.604272 (+ 103us) write_op.cc:276] Timestamp: P: 1786260755604126 usec, L: 0
0809 07:32:35.604292 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 07:32:35.605029 (+ 737us) log.cc:842] Serialized 537 byte log entry
0809 07:32:35.822636 (+217607us) op_driver.cc:444] REPLICATION: finished
0809 07:32:36.001932 (+179296us) write_op.cc:301] APPLY: starting
0809 07:32:36.002092 (+ 160us) tablet.cc:1365] starting BulkCheckPresence
0809 07:32:36.002284 (+ 192us) tablet.cc:1368] finished BulkCheckPresence
0809 07:32:36.002806 (+ 522us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:32:36.003139 (+ 333us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:32:36.003492 (+ 353us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:32:36.003579 (+ 87us) write_op.cc:312] APPLY: finished
0809 07:32:36.004025 (+ 446us) log.cc:842] Serialized 22 byte log entry
0809 07:32:36.004301 (+ 276us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:36.004438 (+ 137us) write_op.cc:454] Released schema lock
0809 07:32:36.004490 (+ 52us) write_op.cc:341] FINISH: Updating metrics
0809 07:32:36.022367 (+ 17877us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:36.022403 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":289312,"reported_tablets":1,"spinlock_wait_cycles":16529920,"child_traces":[["op",{"apply.queue_time_us":178479,"apply.run_cpu_time_us":3249,"apply.run_wall_time_us":20806,"num_ops":1,"prepare.queue_time_us":17726,"prepare.run_cpu_time_us":2990,"prepare.run_wall_time_us":3740,"raft.queue_time_us":521172,"raft.run_cpu_time_us":36009,"raft.run_wall_time_us":211540,"replication_time_us":218236,"thread_start_us":87554,"threads_started":4,"wal-append.queue_time_us":975}]]}
I20260809 07:32:36.047567 31244 tablet_bootstrap.cc:492] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:36.055009 31244 ts_tablet_manager.cc:1397] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.418s user 0.002s sys 0.042s
I20260809 07:32:36.054102 31245 raft_consensus.cc:348] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.058359 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:36.055567 31745 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.063670 31244 raft_consensus.cc:348] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.064455 31244 raft_consensus.cc:374] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.064750 31244 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.064967 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:36.065371 31244 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:36.077879 30853 leader_election.cc:341] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.082118 31244 ts_tablet_manager.cc:1428] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.027s user 0.003s sys 0.003s
W20260809 07:32:36.058873 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 82540.4us
I20260809 07:32:36.055728 31245 raft_consensus.cc:374] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.090984 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:36.091779 31057 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.094755 31244 tablet_bootstrap.cc:492] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:36.089288 31245 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.104614 31243 tablet_bootstrap.cc:492] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.105301 31243 ts_tablet_manager.cc:1397] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.455s user 0.024s sys 0.012s
I20260809 07:32:36.119441 31243 raft_consensus.cc:348] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.120257 31243 raft_consensus.cc:374] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.120563 31243 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.110360 31245 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.121238 31243 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.132133 31057 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:36.143118 31243 ts_tablet_manager.cc:1428] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.037s user 0.005s sys 0.002s
I20260809 07:32:36.144614 31243 tablet_bootstrap.cc:492] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.147500 31245 ts_tablet_manager.cc:1428] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.136s user 0.007s sys 0.002s
W20260809 07:32:36.151090 31006 leader_election.cc:341] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:36.151808 31005 leader_election.cc:341] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.154132 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:36.156836 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 53625.6us
I20260809 07:32:36.157727 31745 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:36.232285 31738 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.233104 31738 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:36.235877 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 138216us
I20260809 07:32:36.238157 31245 tablet_bootstrap.cc:492] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:36.269727 31738 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.270033 31243 tablet_bootstrap.cc:654] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.270834 31245 tablet_bootstrap.cc:654] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.275416 30745 catalog_manager.cc:5495] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 0 to 1, leader changed from <none> to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 1 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:36.279114 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:36.281337 31005 leader_election.cc:341] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.283737 30852 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:32:36.290592 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:36.295867 31815 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:36.296402 31815 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:32:36.301604 30853 consensus_peers.cc:487] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:36.307603 31738 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:36.310714 31848 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:36.297238 31815 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.347067 31244 tablet_bootstrap.cc:654] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.362929 31245 tablet_bootstrap.cc:492] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:36.362887 31174 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.363598 31245 ts_tablet_manager.cc:1397] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.167s user 0.017s sys 0.008s
I20260809 07:32:36.368826 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:36.370545 31245 raft_consensus.cc:348] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.371969 31245 raft_consensus.cc:374] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.372763 31245 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.384572 31245 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.387542 31245 ts_tablet_manager.cc:1428] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.024s user 0.009s sys 0.009s
I20260809 07:32:36.388892 31245 tablet_bootstrap.cc:492] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
W20260809 07:32:36.400527 31006 leader_election.cc:341] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.401046 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:36.402941 31745 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:36.418704 31243 tablet_bootstrap.cc:492] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.421658 31245 tablet_bootstrap.cc:654] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.422358 31243 ts_tablet_manager.cc:1397] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.278s user 0.012s sys 0.032s
I20260809 07:32:36.439594 31244 tablet_bootstrap.cc:492] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:36.440203 31244 ts_tablet_manager.cc:1397] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.346s user 0.020s sys 0.018s
I20260809 07:32:36.446205 31244 raft_consensus.cc:348] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.447037 31244 raft_consensus.cc:374] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.448292 31244 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.448966 31244 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.451337 31244 ts_tablet_manager.cc:1428] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.011s user 0.006s sys 0.001s
I20260809 07:32:36.457144 31244 tablet_bootstrap.cc:492] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:36.458742 31243 raft_consensus.cc:348] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.460872 31243 raft_consensus.cc:374] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.461193 31243 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.462003 31243 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.469389 31243 ts_tablet_manager.cc:1428] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.047s user 0.006s sys 0.000s
I20260809 07:32:36.469102 31835 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 07:32:36.470523 31243 tablet_bootstrap.cc:492] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.472338 30742 catalog_manager.cc:5495] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:36.487190 31817 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.487978 31817 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.489882 31835 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.490653 31835 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.492369 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:36.493305 31051 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:36.495849 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:36.498289 31848 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:36.499224 31005 leader_election.cc:341] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.499053 31848 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.501073 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:36.504324 31835 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.506652 31244 tablet_bootstrap.cc:654] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.505635 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:36.507702 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:36.508389 31006 leader_election.cc:341] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.508812 31006 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:36.509630 31815 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:36.509461 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:36.510214 31815 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:36.510771 31815 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.511996 31835 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:36.512737 31848 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.516572 30852 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:36.519395 31545 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:36.531423 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:36.535820 30853 leader_election.cc:341] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.529054 31817 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.537885 30906 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.563300 31245 tablet_bootstrap.cc:492] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:36.566664 31245 ts_tablet_manager.cc:1397] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.178s user 0.015s sys 0.007s
I20260809 07:32:36.571652 31681 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.572163 31681 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.576617 31681 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:36.583047 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:36.583061 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:36.584550 31120 leader_election.cc:341] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.587584 31815 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.600507 31243 tablet_bootstrap.cc:654] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.605043 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:36.605600 31244 tablet_bootstrap.cc:492] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:36.605688 31058 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.606200 31244 ts_tablet_manager.cc:1397] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.149s user 0.028s sys 0.034s
I20260809 07:32:36.607205 31245 raft_consensus.cc:348] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.607950 31245 raft_consensus.cc:374] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.608129 31245 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.610881 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:36.612434 30853 leader_election.cc:341] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.613032 31815 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.616183 31244 raft_consensus.cc:348] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.616923 31244 raft_consensus.cc:374] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.617226 31244 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.617960 31244 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.620697 31244 ts_tablet_manager.cc:1428] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260809 07:32:36.621869 31244 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:36.624902 31058 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:36.626338 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:36.630496 31815 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:36.631644 31243 tablet_bootstrap.cc:492] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.632228 31243 ts_tablet_manager.cc:1397] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.162s user 0.013s sys 0.003s
I20260809 07:32:36.624822 31245 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.633829 30904 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:36.636371 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:36.637512 31244 tablet_bootstrap.cc:654] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.638501 31681 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:36.638885 31681 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:36.639215 31681 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.638170 31243 raft_consensus.cc:348] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.640378 31243 raft_consensus.cc:374] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.640640 31243 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.641312 31243 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.643659 31243 ts_tablet_manager.cc:1428] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260809 07:32:36.652249 31243 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.631716 31815 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:36.649209 31681 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.664319 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:36.664959 30906 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.675238 31243 tablet_bootstrap.cc:654] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.675537 30906 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:36.655838 31815 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.678418 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:36.679034 31058 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.680747 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:36.681727 31836 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:36.682197 31836 raft_consensus.cc:686] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:36.683116 31836 consensus_queue.cc:237] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.679869 31681 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:36.677088 31245 ts_tablet_manager.cc:1428] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.108s user 0.006s sys 0.001s
I20260809 07:32:36.688853 31245 tablet_bootstrap.cc:492] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:36.693989 31058 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:36.699354 31244 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:36.699891 31244 ts_tablet_manager.cc:1397] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.078s user 0.025s sys 0.015s
I20260809 07:32:36.702371 31244 raft_consensus.cc:348] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.703974 31244 raft_consensus.cc:374] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.704289 31244 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.704988 31244 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.705531 31245 tablet_bootstrap.cc:654] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.707456 31244 ts_tablet_manager.cc:1428] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:32:36.714984 30744 catalog_manager.cc:5495] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:36.723462 31244 tablet_bootstrap.cc:492] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:36.728907 31244 tablet_bootstrap.cc:654] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.734647 31847 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.735095 31847 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.737058 31847 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:36.739722 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:36.741503 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:36.743268 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:36.744344 31681 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:36.749955 30742 catalog_manager.cc:5495] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:36.719132 31243 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.751824 30742 catalog_manager.cc:5495] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:36.766428 31243 ts_tablet_manager.cc:1397] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.114s user 0.016s sys 0.007s
I20260809 07:32:36.776006 31243 raft_consensus.cc:348] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.777038 31243 raft_consensus.cc:374] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.778992 31243 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.779655 31243 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.788080 31932 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.788560 31932 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.791968 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:36.792924 31932 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:36.795969 31005 leader_election.cc:341] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.796661 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:36.797948 31006 leader_election.cc:341] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.795946 31243 ts_tablet_manager.cc:1428] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260809 07:32:36.798373 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:36.799605 31836 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.800084 31836 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.800700 31932 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:36.801839 31243 tablet_bootstrap.cc:492] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.804234 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:36.805864 31120 leader_election.cc:341] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.806615 31244 tablet_bootstrap.cc:492] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:36.807051 31244 ts_tablet_manager.cc:1397] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.084s user 0.011s sys 0.030s
I20260809 07:32:36.802078 31836 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:36.807301 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:36.810214 31243 tablet_bootstrap.cc:654] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.810999 31058 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:36.812567 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:36.804498 31245 tablet_bootstrap.cc:492] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:36.813390 31836 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:36.813900 31836 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:36.809439 31244 raft_consensus.cc:348] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.814898 31244 raft_consensus.cc:374] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.815176 31244 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.815793 31244 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.816591 31836 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.817955 31244 ts_tablet_manager.cc:1428] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260809 07:32:36.819689 31244 tablet_bootstrap.cc:492] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:36.824488 31836 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.826597 31836 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:36.813956 31245 ts_tablet_manager.cc:1397] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.125s user 0.024s sys 0.020s
I20260809 07:32:36.830123 31245 raft_consensus.cc:348] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.830921 31245 raft_consensus.cc:374] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.831225 31245 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.833379 31243 tablet_bootstrap.cc:492] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.833885 31243 ts_tablet_manager.cc:1397] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.008s
I20260809 07:32:36.834506 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:36.835078 31058 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.825103 31244 tablet_bootstrap.cc:654] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.837564 31243 raft_consensus.cc:348] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.852322 31243 raft_consensus.cc:374] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.853731 31243 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.854815 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:36.852970 31245 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.854645 31243 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:36.859802 31120 leader_election.cc:341] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.868827 31244 tablet_bootstrap.cc:492] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:36.869853 31244 ts_tablet_manager.cc:1397] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.050s user 0.008s sys 0.013s
I20260809 07:32:36.869980 31243 ts_tablet_manager.cc:1428] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.036s user 0.003s sys 0.004s
I20260809 07:32:36.871131 31243 tablet_bootstrap.cc:492] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.872283 31244 raft_consensus.cc:348] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.872975 31244 raft_consensus.cc:374] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.873217 31244 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.877451 31243 tablet_bootstrap.cc:654] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.887106 31245 ts_tablet_manager.cc:1428] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.059s user 0.004s sys 0.003s
I20260809 07:32:36.873946 31244 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.882757 31058 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:36.889000 31244 ts_tablet_manager.cc:1428] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260809 07:32:36.889959 31244 tablet_bootstrap.cc:492] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:36.892637 31165 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:36.895175 31244 tablet_bootstrap.cc:654] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.895087 31245 tablet_bootstrap.cc:492] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:36.911077 31245 tablet_bootstrap.cc:654] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:36.914098 30853 consensus_peers.cc:487] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:36.921217 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:36.924198 31243 tablet_bootstrap.cc:492] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.924656 31243 ts_tablet_manager.cc:1397] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.054s user 0.015s sys 0.000s
I20260809 07:32:36.927047 31243 raft_consensus.cc:348] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.927700 31243 raft_consensus.cc:374] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.927969 31243 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.928470 31681 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:36.929008 31681 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:36.928592 31243 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.929801 31681 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.930377 31243 ts_tablet_manager.cc:1428] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260809 07:32:36.935462 31243 tablet_bootstrap.cc:492] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.940194 31545 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:36.941054 31243 tablet_bootstrap.cc:654] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:36.974651 31932 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.974429 30742 catalog_manager.cc:5495] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:36.975188 31932 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.977447 31932 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:36.978595 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:36.979732 31005 leader_election.cc:341] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:36.985103 31243 tablet_bootstrap.cc:492] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:36.985616 31243 ts_tablet_manager.cc:1397] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.050s user 0.014s sys 0.005s
I20260809 07:32:36.988061 31243 raft_consensus.cc:348] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.988709 31243 raft_consensus.cc:374] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:36.988970 31243 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:36.989586 31243 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.991773 31243 ts_tablet_manager.cc:1428] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:32:36.992800 31243 tablet_bootstrap.cc:492] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:36.994768 31836 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:36.995265 31836 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:36.997170 31836 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:36.998212 31243 tablet_bootstrap.cc:654] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.000263 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:37.014703 31243 tablet_bootstrap.cc:492] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:37.015218 31243 ts_tablet_manager.cc:1397] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.002s
I20260809 07:32:37.017761 31243 raft_consensus.cc:348] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.018777 31243 raft_consensus.cc:374] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.019332 31243 raft_consensus.cc:729] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.019954 31243 consensus_queue.cc:260] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.027658 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.038938 31932 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.039629 31932 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.090034 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:37.112596 31005 leader_election.cc:341] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.144657 31681 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.145205 31681 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.147202 31681 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:37.177722 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 52885.1us
I20260809 07:32:37.182978 31932 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.227159 31245 tablet_bootstrap.cc:492] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:37.227670 31245 ts_tablet_manager.cc:1397] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.333s user 0.073s sys 0.095s
I20260809 07:32:37.227202 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:37.241382 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 89428.2us
I20260809 07:32:37.242858 31243 ts_tablet_manager.cc:1428] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.227s user 0.083s sys 0.106s
I20260809 07:32:37.244671 31243 tablet_bootstrap.cc:492] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:37.252220 31245 raft_consensus.cc:348] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.252971 31245 raft_consensus.cc:374] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.253242 31245 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.253806 31245 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.256047 31245 ts_tablet_manager.cc:1428] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260809 07:32:37.256891 31245 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
W20260809 07:32:37.265293 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 62173.5us
I20260809 07:32:37.272905 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:37.274094 31988 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:37.270639 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:37.309865 31995 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.310968 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:37.317950 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 67613.7us
I20260809 07:32:37.321761 31245 tablet_bootstrap.cc:654] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.302245 31815 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
W20260809 07:32:37.334969 31006 leader_election.cc:341] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.335613 31006 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.318531 31995 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.337280 31835 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:37.338970 31995 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.339258 31988 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.346457 31988 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.348384 31988 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:37.350054 31244 tablet_bootstrap.cc:492] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
W20260809 07:32:37.340902 31006 leader_election.cc:341] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.352010 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.357816 31244 ts_tablet_manager.cc:1397] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.468s user 0.055s sys 0.088s
I20260809 07:32:37.360564 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:37.361340 31120 leader_election.cc:341] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:37.371325 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 81689us
I20260809 07:32:37.376159 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.376787 31058 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:37.377666 31244 raft_consensus.cc:348] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.378494 31244 raft_consensus.cc:374] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.378753 31244 raft_consensus.cc:729] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.379357 31244 consensus_queue.cc:260] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.377544 31932 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:37.385504 31244 ts_tablet_manager.cc:1428] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260809 07:32:37.387774 31244 tablet_bootstrap.cc:492] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
W20260809 07:32:37.380858 31120 leader_election.cc:341] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.415025 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:37.418022 31243 tablet_bootstrap.cc:654] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
W20260809 07:32:37.419939 30853 leader_election.cc:341] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
W20260809 07:32:37.421846 30853 consensus_peers.cc:487] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:37.405678 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.427879 31048 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
W20260809 07:32:37.428495 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:37.431742 31990 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.418848 32010 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260809 07:32:37.432368 31990 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.436189 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:37.438261 31990 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:37.442267 31120 leader_election.cc:341] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.447407 30852 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:37.457803 32015 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:37.458210 32015 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:37.458560 32015 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.478400 31245 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:37.478935 31245 ts_tablet_manager.cc:1397] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.222s user 0.028s sys 0.032s
I20260809 07:32:37.482614 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.482766 31245 raft_consensus.cc:348] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.483227 31058 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:37.483381 31245 raft_consensus.cc:374] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.483628 31245 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.484197 31245 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.488258 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.503000 31245 ts_tablet_manager.cc:1428] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20260809 07:32:37.510210 31244 tablet_bootstrap.cc:654] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.499044 31988 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:37.504894 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.515213 31243 tablet_bootstrap.cc:492] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:37.515676 31243 ts_tablet_manager.cc:1397] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.272s user 0.017s sys 0.026s
I20260809 07:32:37.517999 31245 tablet_bootstrap.cc:492] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:37.513603 31988 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:37.523766 31988 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.513033 31057 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:37.531244 31243 raft_consensus.cc:348] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.532225 31243 raft_consensus.cc:374] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.532568 31243 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.533262 31243 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.535748 32015 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.537729 32015 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.538479 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:37.539062 31046 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.539726 31243 ts_tablet_manager.cc:1428] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
I20260809 07:32:37.545306 31165 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.553850 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:37.561301 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 55984us
I20260809 07:32:37.563669 32010 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:37.565292 31243 tablet_bootstrap.cc:492] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:37.565476 31120 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.568729 31244 tablet_bootstrap.cc:492] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:37.569207 31244 ts_tablet_manager.cc:1397] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.182s user 0.013s sys 0.015s
I20260809 07:32:37.572113 31244 raft_consensus.cc:348] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.572764 31244 raft_consensus.cc:374] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.572929 31245 tablet_bootstrap.cc:654] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.572966 31244 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.576406 31046 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:37.575526 31244 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.579151 30852 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:37.580701 31992 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 07:32:37.581059 30853 leader_election.cc:341] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.581135 31992 raft_consensus.cc:686] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:37.582016 31992 consensus_queue.cc:237] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.583060 31988 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.585816 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.586629 31980 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:37.586966 31980 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:37.589237 31244 ts_tablet_manager.cc:1428] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.020s user 0.004s sys 0.003s
I20260809 07:32:37.587319 31980 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.596412 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:37.597051 31058 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.598541 31847 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 07:32:37.598973 31244 tablet_bootstrap.cc:492] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Bootstrap starting.
I20260809 07:32:37.584849 31988 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:37.614539 31244 tablet_bootstrap.cc:654] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.587170 32028 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:37.615698 30900 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.623988 32028 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:37.621466 31980 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.622448 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:37.627241 32028 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.630825 31932 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.631302 31932 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.633148 31932 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.634227 31058 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:37.636658 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:37.638988 31120 leader_election.cc:341] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.639832 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:37.640913 31006 leader_election.cc:341] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.638509 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:37.642740 31005 leader_election.cc:341] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.643114 31005 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.644068 32001 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:37.650781 31987 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:37.667510 31980 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:37.687389 31987 raft_consensus.cc:686] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:37.688606 31987 consensus_queue.cc:237] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.689471 30744 catalog_manager.cc:5495] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:37.693647 31991 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:37.707527 31245 tablet_bootstrap.cc:492] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:37.707938 31245 ts_tablet_manager.cc:1397] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.200s user 0.018s sys 0.011s
I20260809 07:32:37.711252 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:37.716576 31057 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.717049 31245 raft_consensus.cc:348] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.717746 31245 raft_consensus.cc:374] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.692375 32028 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.718118 31245 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.718842 31245 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.710143 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:37.722442 32028 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:37.724033 31245 ts_tablet_manager.cc:1428] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260809 07:32:37.725134 31245 tablet_bootstrap.cc:492] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:37.727334 31996 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.727793 31996 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:37.741293 31120 leader_election.cc:341] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.747514 31996 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.748401 31243 tablet_bootstrap.cc:654] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.744624 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:37.749379 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:37.749115 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:37.758802 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.772866 31244 tablet_bootstrap.cc:492] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: No bootstrap required, opened a new log
I20260809 07:32:37.773331 31244 ts_tablet_manager.cc:1397] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Time spent bootstrapping tablet: real 0.175s user 0.011s sys 0.008s
I20260809 07:32:37.773514 31051 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:37.765308 31046 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:32:37.778040 31120 leader_election.cc:341] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.778954 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:37.785348 30853 leader_election.cc:341] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.785104 31244 raft_consensus.cc:348] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.787097 31244 raft_consensus.cc:374] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.787371 31244 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.796137 31993 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:37.796519 31993 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:37.796666 31245 tablet_bootstrap.cc:654] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.796888 31993 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.798949 31244 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.814056 31244 ts_tablet_manager.cc:1428] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Time spent starting tablet: real 0.040s user 0.007s sys 0.000s
I20260809 07:32:37.824769 31057 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:37.829680 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.831403 31991 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:37.833165 31991 raft_consensus.cc:686] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:32:37.834478 32035 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
I20260809 07:32:37.834029 31991 consensus_queue.cc:237] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.837808 31243 tablet_bootstrap.cc:492] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:37.838299 31243 ts_tablet_manager.cc:1397] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.290s user 0.020s sys 0.014s
I20260809 07:32:37.841895 31243 raft_consensus.cc:348] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.842644 31243 raft_consensus.cc:374] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.843174 31243 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.845933 32028 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:37.846413 32028 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.846637 31993 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.855592 32028 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:37.840554 31046 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:37.900388 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:37.900748 31243 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.860950 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.901095 31048 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.893137 31993 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.902724 31245 tablet_bootstrap.cc:492] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:37.903187 31245 ts_tablet_manager.cc:1397] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.178s user 0.016s sys 0.007s
I20260809 07:32:37.856971 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:37.914695 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:37.916008 31048 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
W20260809 07:32:37.888871 32037 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.072s user 0.001s sys 0.000s
I20260809 07:32:37.919275 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:37.911089 31245 raft_consensus.cc:348] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.921240 31245 raft_consensus.cc:374] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:37.921561 31245 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:37.904299 31993 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:37.923381 30853 leader_election.cc:341] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:37.923666 31245 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:37.927568 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 56655.8us
I20260809 07:32:37.923185 32015 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:37.930014 31120 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:37.931854 31243 ts_tablet_manager.cc:1428] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.093s user 0.011s sys 0.021s
I20260809 07:32:37.932900 31991 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:37.933084 31245 ts_tablet_manager.cc:1428] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.030s user 0.007s sys 0.001s
I20260809 07:32:37.935396 32015 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:37.937726 32015 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.941509 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:37.945996 31988 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:37.946291 31243 tablet_bootstrap.cc:492] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:37.933686 31991 raft_consensus.cc:686] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:37.952164 31174 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:37.952003 31991 consensus_queue.cc:237] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.954454 31243 tablet_bootstrap.cc:654] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:37.958212 31245 tablet_bootstrap.cc:492] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:37.927279 31993 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:37.968106 30745 catalog_manager.cc:5495] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:37.971954 31993 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:37.973402 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:37.974227 31048 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:37.985708 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:37.998400 31847 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.006624 31847 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.012025 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:38.012737 31992 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.013936 31815 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.014485 31815 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.011592 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:38.019565 31815 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.020597 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:38.021236 31165 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:38.022351 30853 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:38.023263 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.024888 31048 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:38.027869 31847 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.035612 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.060201 31243 tablet_bootstrap.cc:492] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:38.067953 31048 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:38.061866 31245 tablet_bootstrap.cc:654] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.068089 31243 ts_tablet_manager.cc:1397] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.122s user 0.010s sys 0.030s
I20260809 07:32:38.027236 30909 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:38.069578 31815 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:38.071346 31815 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:38.071683 31815 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.074060 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:38.075914 31243 raft_consensus.cc:348] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.076613 31243 raft_consensus.cc:374] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.076853 31243 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.078131 32028 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:38.077687 31243 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.078692 32028 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:38.079573 31992 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.079918 32028 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.087723 31243 ts_tablet_manager.cc:1428] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.019s user 0.000s sys 0.005s
I20260809 07:32:38.088689 31243 tablet_bootstrap.cc:492] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:38.094015 31243 tablet_bootstrap.cc:654] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.097826 31815 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.098681 32028 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.100250 32028 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.101076 31815 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.100957 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:38.101779 30900 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:38.100509 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:38.103243 31048 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.102699 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:38.104094 31174 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:38.104018 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:38.106912 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:38.107955 31847 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.112746 30742 catalog_manager.cc:5495] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:38.123250 30745 catalog_manager.cc:5495] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:38.125250 30745 catalog_manager.cc:5495] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:38.133982 31245 tablet_bootstrap.cc:492] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:38.134516 31245 ts_tablet_manager.cc:1397] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.179s user 0.006s sys 0.007s
I20260809 07:32:38.134445 31048 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:38.143308 30852 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:38.144374 31815 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:38.144816 31815 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:38.145548 31815 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.151753 31245 raft_consensus.cc:348] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.152427 31245 raft_consensus.cc:374] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.152648 31245 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.153175 31245 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.157348 32028 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.185379 31243 tablet_bootstrap.cc:492] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:38.185839 31243 ts_tablet_manager.cc:1397] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.097s user 0.014s sys 0.001s
I20260809 07:32:38.187207 31993 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.192024 31243 raft_consensus.cc:348] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.192768 31243 raft_consensus.cc:374] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.193017 31243 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.194628 31245 ts_tablet_manager.cc:1428] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.060s user 0.006s sys 0.000s
I20260809 07:32:38.195703 31243 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.202579 31245 tablet_bootstrap.cc:492] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:38.203472 31174 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.205829 31243 ts_tablet_manager.cc:1428] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.020s user 0.003s sys 0.003s
I20260809 07:32:38.219542 32001 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.220088 32001 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.224021 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:38.226006 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:38.227710 31005 leader_election.cc:341] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:38.225507 30745 catalog_manager.cc:5495] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:32:38.233537 31006 leader_election.cc:341] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:38.233994 31006 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:38.234701 31930 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.222052 32001 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.261174 31243 tablet_bootstrap.cc:492] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:38.266482 31815 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.278434 31988 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.289881 31815 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.292955 31815 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.294970 31815 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.295691 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.296573 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:38.296917 31057 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:38.298867 30852 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:38.300601 31243 tablet_bootstrap.cc:654] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.302170 31815 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.306164 31245 tablet_bootstrap.cc:654] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.335938 31815 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.340692 31243 tablet_bootstrap.cc:492] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:38.341255 31243 ts_tablet_manager.cc:1397] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.081s user 0.012s sys 0.001s
I20260809 07:32:38.341442 31245 tablet_bootstrap.cc:492] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:38.341900 31245 ts_tablet_manager.cc:1397] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.145s user 0.004s sys 0.007s
I20260809 07:32:38.345326 31245 raft_consensus.cc:348] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.345973 31245 raft_consensus.cc:374] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.346246 31245 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.348249 31243 raft_consensus.cc:348] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.349016 31243 raft_consensus.cc:374] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.351020 31245 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.356703 31243 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.358800 31245 ts_tablet_manager.cc:1428] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260809 07:32:38.358011 31243 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.359795 31245 tablet_bootstrap.cc:492] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:38.361387 31243 ts_tablet_manager.cc:1428] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260809 07:32:38.362275 31243 tablet_bootstrap.cc:492] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:38.365785 31245 tablet_bootstrap.cc:654] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.369935 31243 tablet_bootstrap.cc:654] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.387168 31243 tablet_bootstrap.cc:492] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:38.387692 31243 ts_tablet_manager.cc:1397] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.000s
I20260809 07:32:38.390177 31243 raft_consensus.cc:348] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.390836 31243 raft_consensus.cc:374] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.391183 31243 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.391692 31243 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.393455 31243 ts_tablet_manager.cc:1428] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:32:38.394975 31243 tablet_bootstrap.cc:492] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Bootstrap starting.
I20260809 07:32:38.397507 31245 tablet_bootstrap.cc:492] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:38.397975 31245 ts_tablet_manager.cc:1397] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.012s
I20260809 07:32:38.400552 31243 tablet_bootstrap.cc:654] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.400925 31245 raft_consensus.cc:348] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.407331 31245 raft_consensus.cc:374] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.407588 31245 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.408310 31245 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.411357 31245 ts_tablet_manager.cc:1428] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260809 07:32:38.412412 31243 tablet_bootstrap.cc:492] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: No bootstrap required, opened a new log
I20260809 07:32:38.412879 31243 ts_tablet_manager.cc:1397] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.003s
I20260809 07:32:38.412983 31245 tablet_bootstrap.cc:492] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:38.418780 31245 tablet_bootstrap.cc:654] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.419068 31847 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.419523 31847 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.420409 31174 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.421475 31243 raft_consensus.cc:348] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.422153 31243 raft_consensus.cc:374] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.422421 31243 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.422812 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:38.423022 31243 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.425349 31243 ts_tablet_manager.cc:1428] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260809 07:32:38.425190 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.425989 31048 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:38.421401 31847 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.427457 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:38.429562 31120 consensus_peers.cc:487] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 -> Peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Couldn't send request to peer 2c9de56c3fa3487c9dac990675c34be5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:38.432502 32028 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.432969 32028 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.434854 32028 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.436694 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.437392 31048 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:38.439494 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:38.442732 31847 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.446218 31993 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:32:38.457347 31120 leader_election.cc:341] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:38.458166 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:38.460850 31847 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:38.461226 31847 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:38.461591 31847 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.468770 31988 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.483533 31815 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.496188 31991 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.472333 31847 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.505314 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:32:38.507803 31120 leader_election.cc:341] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Illegal state: must be running to vote when last-logged opid is not known
I20260809 07:32:38.516678 31847 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.518678 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:38.519296 31057 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.529211 31057 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:38.541828 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:38.542773 31988 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:38.543377 31988 raft_consensus.cc:686] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:38.544298 31988 consensus_queue.cc:237] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:38.566570 30827 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 07:32:38.568049 31993 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.589668 31245 tablet_bootstrap.cc:492] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:38.590209 31245 ts_tablet_manager.cc:1397] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.177s user 0.041s sys 0.030s
I20260809 07:32:38.591411 31988 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.593044 31245 raft_consensus.cc:348] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.593899 31245 raft_consensus.cc:374] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.594342 31245 raft_consensus.cc:729] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.595003 31245 consensus_queue.cc:260] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.597704 31245 ts_tablet_manager.cc:1428] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 07:32:38.605150 31993 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:38.608085 31245 tablet_bootstrap.cc:492] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: Bootstrap starting.
I20260809 07:32:38.613931 31245 tablet_bootstrap.cc:654] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: Neither blocks nor log segments found. Creating new log.
I20260809 07:32:38.651940 31993 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.652487 31993 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.663679 31993 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.705577 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.706379 31048 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
W20260809 07:32:38.714478 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 45529.4us
I20260809 07:32:38.719982 31991 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.720427 31991 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.722249 31991 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.727048 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:38.728694 31815 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:38.729051 31815 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:38.729367 31815 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.737249 31245 tablet_bootstrap.cc:492] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: No bootstrap required, opened a new log
I20260809 07:32:38.739748 31815 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.744581 31815 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.737867 31245 ts_tablet_manager.cc:1397] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: Time spent bootstrapping tablet: real 0.130s user 0.012s sys 0.005s
I20260809 07:32:38.759450 31245 raft_consensus.cc:348] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.775580 31245 raft_consensus.cc:374] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:32:38.761315 31815 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.762852 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:38.762858 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.765352 31987 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.778192 31987 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.778393 31815 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.780161 31987 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.768877 32015 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:38.780499 31815 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.780783 32015 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.771564 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:38.782572 31181 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:38.772665 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:38.783355 31164 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.778203 30909 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:38.782224 32015 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.788018 31046 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:38.788832 31164 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:38.789122 30744 catalog_manager.cc:5495] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:38.777644 31245 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Initialized, Role: FOLLOWER
I20260809 07:32:38.794768 31245 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.811983 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:38.816293 30909 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:38.814678 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.817426 31046 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 0.
I20260809 07:32:38.818935 30853 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:38.825215 32015 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:38.825971 32015 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:38.826954 32015 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.839955 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:38.841086 31987 raft_consensus.cc:2741] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 07:32:38.841954 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:38.845036 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 116507us
I20260809 07:32:38.845908 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:38.846537 31847 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:38.846931 31847 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:38.847230 31847 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.847746 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:38.848255 31057 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.849578 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.848102 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:38.852403 31245 ts_tablet_manager.cc:1428] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: Time spent starting tablet: real 0.095s user 0.005s sys 0.001s
I20260809 07:32:38.854235 31048 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 0.
I20260809 07:32:38.854260 31847 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.856702 31847 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:38.857359 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:38.858171 31164 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:38.859459 31057 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:38.867340 30852 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:38.868503 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:38.869213 31815 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.869477 32097 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:32:38.869897 32097 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:38.870216 32097 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.877015 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:38.879065 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:38.879854 31057 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:38.899062 31057 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:38.912807 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:38.914469 31990 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:32:38.914956 31990 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:38.915714 31990 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.921219 30744 catalog_manager.cc:5495] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 1 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:38.929076 32097 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:38.931241 30904 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 1: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:38.931977 32097 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:38.935567 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:38.942934 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:38.943691 31057 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 1: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:38.944558 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:38.945220 31815 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 07:32:38.952469 30744 catalog_manager.cc:5495] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 1 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:39.009686 32028 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:39.028004 30904 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:39.044855 30909 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:32:39.052754 32028 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:39.326118 31990 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:39.355768 31993 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:39.358982 31991 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:32:39.375550 32097 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 07:32:40.298998 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:32:40.340238 32122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:40.341832 32127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:32:40.342909 32123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:32:40.345927 29021 server_base.cc:1034] running on GCE node
I20260809 07:32:40.352120 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 07:32:40.352500 29021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 07:32:40.352762 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786260760352743 us; error 0 us; skew 500 ppm
I20260809 07:32:40.353577 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 07:32:40.360538 29021 webserver.cc:460] Webserver started at http://127.28.87.68:33505/ using document root <none> and password file <none>
I20260809 07:32:40.361097 29021 fs_manager.cc:359] Metadata directory not provided
I20260809 07:32:40.361276 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:32:40.361552 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:32:40.362789 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-3-root/instance:
uuid: "28b9965e21374d26b0ac2014c9bafdff"
format_stamp: "Formatted at 2026-08-09 07:32:40 on dist-test-slave-fgb3"
I20260809 07:32:40.369740 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 07:32:40.373796 32135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:40.375345 29021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:32:40.375677 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-3-root
uuid: "28b9965e21374d26b0ac2014c9bafdff"
format_stamp: "Formatted at 2026-08-09 07:32:40 on dist-test-slave-fgb3"
I20260809 07:32:40.376009 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:32:40.394971 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:32:40.396699 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:32:40.398841 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:32:40.401854 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:32:40.402100 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:32:40.402429 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:32:40.402627 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260809 07:32:40.689797 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 202060us
W20260809 07:32:40.993847 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 243971us
W20260809 07:32:41.020970 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 203952us
I20260809 07:32:41.990566 32253 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:41.991987 32253 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:41.996563 32253 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.000504 32253 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.001557 32253 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.006140 32253 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.008440 32256 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.008576 32253 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.009792 32256 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.010422 32253 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.015753 32253 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.017258 32258 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.016973 32253 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.018898 32258 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.020229 32254 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.021351 32256 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.022298 32256 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.021721 32254 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.023216 32256 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.025586 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 461) took 1012 ms. Trace:
I20260809 07:32:42.026127 32258 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.026479 32254 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.027783 32256 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.029059 32256 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.019354 32253 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.030465 32256 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.033814 32253 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.034739 32256 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.029006 31046 rpcz_store.cc:276] 0809 07:32:41.013280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:41.015885 (+ 2605us) service_pool.cc:227] Handling call
0809 07:32:41.035384 (+ 19499us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:41.051596 (+ 16212us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:41.053621 (+ 2025us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:42.024672 (+971051us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:42.024830 (+ 158us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:42.025006 (+ 176us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:42.025374 (+ 368us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:42.060443 32253 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.060961 32261 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.065649 32263 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.065383 32261 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.061175 32258 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.066938 32263 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.068753 32258 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.072192 32263 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.073132 32258 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.062034 32253 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.079118 32253 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.077013 32261 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.080734 32258 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.062223 32256 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.081719 32258 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.082965 32256 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.086364 32258 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.086035 32265 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.087260 32256 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.087577 32265 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.073275 32263 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.093075 32263 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.094214 32249 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.095340 32249 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.097817 32263 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:42.098909 29021 thread.cc:636] result-tracker (server) Time spent creating pthread: real 1.300s user 0.025s sys 0.021s
W20260809 07:32:42.099624 29021 thread.cc:602] result-tracker (server) Time spent starting thread: real 1.301s user 0.026s sys 0.021s
I20260809 07:32:42.099884 32249 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.101235 32265 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.108731 32266 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.109961 32266 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.119616 32266 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.120749 32266 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.121656 32266 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.125113 32266 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.126612 32269 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:42.127713 32269 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.131310 32269 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.133765 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.87.68:39299
I20260809 07:32:42.133811 32269 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:42.135025 32268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.87.68:39299 every 8 connection(s)
I20260809 07:32:42.135403 32269 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.139889 32269 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:42.145002 31003 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.123s user 0.009s sys 0.012s
W20260809 07:32:42.145769 31003 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.124s user 0.009s sys 0.012s
I20260809 07:32:42.147161 32265 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.148092 32265 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.155431 32264 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.156643 32264 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.161303 32256 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.161960 32264 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.162658 32254 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.163801 32254 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.156356 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.332s user 0.010s sys 0.005s
I20260809 07:32:42.167958 32254 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:42.170547 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.346s user 0.010s sys 0.006s
I20260809 07:32:42.156360 32261 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.172629 32261 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.175076 32256 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.180558 32261 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.182525 32256 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.162689 32265 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.199457 32265 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.200116 29021 auto_leader_rebalancer-test.cc:413] add a tserver: 28b9965e21374d26b0ac2014c9bafdff
I20260809 07:32:42.200695 32265 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.201865 30849 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.352s user 0.005s sys 0.009s
I20260809 07:32:42.205108 32278 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.205978 32265 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.206185 32278 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.209944 30849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.362s user 0.007s sys 0.010s
I20260809 07:32:42.215010 32277 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.216127 32277 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.226562 32278 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.233717 32265 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.236557 32265 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.240166 32256 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.247231 32277 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.247722 32261 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.248361 32258 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.255560 32258 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.256508 32265 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.248821 32261 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.261754 32261 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.241328 32256 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.264729 32258 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.271093 32265 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.272019 32261 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.269160 32256 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.274199 32256 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.273741 32261 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.275219 32256 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.273032 32265 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.281107 32258 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.284571 32266 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.282150 32258 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.285101 32261 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.302214 32273 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.296098 32277 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.284907 32265 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.298395 32270 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.313635 32249 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.310422 32277 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.314805 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.491s user 0.016s sys 0.032s
W20260809 07:32:42.317858 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.494s user 0.017s sys 0.032s
I20260809 07:32:42.314759 32249 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.318497 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 1.49889e+06us
I20260809 07:32:42.321470 32269 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.322341 32277 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.304526 32266 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.325368 32249 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.315316 32270 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.309432 32256 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.331928 32277 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.311070 32273 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.330263 32269 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.297240 32285 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.326552 32249 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:32:42.339000 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 71330.4us
I20260809 07:32:42.341235 32277 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.341328 32286 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.347186 32270 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.330443 32265 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:42.345690 32249 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.359004 32286 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.359612 32265 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.357813 32277 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.362525 32277 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.363541 32277 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.364892 32258 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.367350 32273 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.345336 32285 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.372764 32285 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:42.404986 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 85394.5us
I20260809 07:32:42.412122 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.417868 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.392804 32269 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.416695 32265 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.413471 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.421653 31182 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.401196 32277 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.423907 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.374058 32285 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.425329 31170 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.395339 32286 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.419505 32266 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.424165 32277 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.429147 32277 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.431329 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.431777 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.433351 31182 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.420423 32249 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.430025 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.433116 31174 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.435524 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.437901 31172 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.425940 32285 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.439142 32288 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.440327 32288 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.434474 32286 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.434655 32289 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.734659 32289 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.737349 32285 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.739621 32285 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:42.730654 32301 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:32:42.656649 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 226800us
W20260809 07:32:42.683643 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 257050us
I20260809 07:32:42.731078 32286 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.729629 32277 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.646545 32288 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.755033 32301 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.760875 32301 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.761981 32286 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.761201 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.740834 32302 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.741189 32289 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.729492 32300 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.436760 32269 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:42.743270 32303 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:42.671551 32297 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.749549 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.773568 32253 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.783947 32253 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.787173 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.774278 32286 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.774148 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.792012 32286 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.794281 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.795609 31177 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.796306 32286 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.789239 32253 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.798581 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.800359 31164 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.763938 32285 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.777688 32263 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.780827 31165 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.779944 32300 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.806696 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.786s user 0.010s sys 0.011s
I20260809 07:32:42.805908 32263 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.809388 32300 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.781908 32269 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.811321 32263 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.813357 32269 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:42.797334 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.819540 31177 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.784793 32297 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.824808 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.805s user 0.010s sys 0.017s
I20260809 07:32:42.835533 32285 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.787611 32302 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:42.841658 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 93300.9us
I20260809 07:32:42.841385 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.843806 32302 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:42.847332 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 97872us
I20260809 07:32:42.861097 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:42.863431 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:42.864784 31058 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:42.867434 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:42.870391 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:42.848703 32297 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.786938 32303 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.881873 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:42.883785 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:42.853276 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.885432 30906 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:42.891758 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:42.894359 31118 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.844s user 0.036s sys 0.060s
W20260809 07:32:42.894936 31118 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.845s user 0.036s sys 0.060s
I20260809 07:32:42.899516 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.900756 31165 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:42.914076 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:42.926729 32303 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.929491 32306 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:42.930724 32297 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:42.966922 32306 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.970609 32297 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:42.988018 32305 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.010026 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.012629 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.010839 32292 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.015749 31165 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.021855 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.026491 31057 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:43.025372 32269 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.026026 32292 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.029315 32269 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.033051 32306 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:42.989472 32305 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.036325 32297 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.038977 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.041584 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.046446 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.048795 32292 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.050282 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.051115 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.051807 31170 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.052546 31054 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.054992 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.056412 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.057453 31181 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.026243 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.060097 32305 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.046595 32269 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.061431 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.063335 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.079246 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.064556 32239 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.096369 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.095966 32239 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.096707 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.064988 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.107515 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.065809 32249 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.108824 31174 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.109144 32249 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.110270 32269 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.111204 32269 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.113183 32249 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.067389 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.115109 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.068812 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.116678 31169 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.072337 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.119124 31165 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.075237 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.081283 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.063493 32235 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.121501 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.121682 32235 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.122726 31045 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.082934 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.083781 32234 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.084403 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.086581 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.086992 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.086992 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.130806 31055 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:43.090049 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.091185 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.094121 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.101256 30911 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:43.090663 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.105739 32239 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.099455 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.139772 31049 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:43.137238 31175 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.142763 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.132907 32234 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.114109 32249 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.130654 32288 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.139580 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.143776 32235 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.150969 31051 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:43.151211 31046 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.151983 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.154910 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.155731 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.156458 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.160557 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.162774 31059 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.173405 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.178697 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.179332 32235 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.180258 32235 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.181319 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.156381 32288 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.183341 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.190366 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.154912 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.158576 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.164248 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.200776 31054 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.165902 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.204877 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.168414 32269 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.209385 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.211668 32304 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.189236 32277 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.186259 32288 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.189538 32306 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.187337 32230 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.168778 32249 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.213020 32304 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.345002 32234 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.358484 32263 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.362987 32306 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.322190 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.364806 32263 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.356292 32253 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.323555 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.231953 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.325083 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.415123 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.356041 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.331496 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.328193 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.284226 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.324388 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.323522 32235 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.336937 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.319718 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.305372 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.427049 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.356036 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.427826 31049 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.428143 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.373206 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.430580 31172 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.416503 32253 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.330776 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:43.299703 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 107953us
I20260809 07:32:43.307893 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.426896 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.236709 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.403980 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.369737 32304 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.248663 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.408479 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.332851 32001 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.410823 32263 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.362493 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.293814 32265 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.423365 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.447659 31177 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.429731 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.453033 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.463910 32253 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.453711 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:43.452805 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.471774 32249 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.457374 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.455291 32001 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.460304 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.458942 32265 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.484723 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.478751 32235 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.488598 32265 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.489266 32235 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:32:43.284582 31215 debug-util.cc:398] Leaking SignalData structure 0x7b08001390e0 after lost signal to thread 29026
I20260809 07:32:43.425230 32230 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.489817 31054 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:43.418025 32277 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.496965 32277 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.497900 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.498617 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.493988 32230 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.497792 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.500563 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.488631 32001 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.490427 32235 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.508834 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:43.510286 31215 debug-util.cc:398] Leaking SignalData structure 0x7b080013bf40 after lost signal to thread 30803
W20260809 07:32:43.513234 31215 debug-util.cc:398] Leaking SignalData structure 0x7b08002e70c0 after lost signal to thread 30976
I20260809 07:32:43.492185 32306 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.516526 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:43.517935 32309 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.518638 32309 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:43.519507 32309 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.524216 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:43.535378 31215 debug-util.cc:398] Leaking SignalData structure 0x7b08002e7900 after lost signal to thread 31100
I20260809 07:32:43.550729 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.551565 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.554890 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.558102 32265 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.558964 32265 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.559794 32265 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.560185 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.562606 32271 heartbeater.cc:347] Connected to a master server at 127.28.87.126:41365
I20260809 07:32:43.563409 32271 heartbeater.cc:464] Registering TS with master...
I20260809 07:32:43.564347 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.565366 32271 heartbeater.cc:511] Master 127.28.87.126:41365 requested a full tablet report, sending...
I20260809 07:32:43.565837 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.566545 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.568514 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:43.568972 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.569514 32001 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.576622 30744 ts_manager.cc:194] Registered new tserver with Master: 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:32:43.577848 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.585268 32001 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.587024 32001 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.579030 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.591101 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.591987 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.579262 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.582144 32265 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.595634 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.596145 32309 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.597101 30900 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.601464 30744 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48008
W20260809 07:32:43.572409 31215 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed140 after lost signal to thread 31219
W20260809 07:32:43.602492 31215 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed160 after lost signal to thread 31220
I20260809 07:32:43.580323 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.603464 30852 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.605387 32306 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.606099 32306 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.606920 32306 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.609333 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.613912 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.618045 30852 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.618750 32263 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.634256 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.634589 32263 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.635592 32263 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.636610 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:43.619454 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 50640.6us
I20260809 07:32:43.638923 31006 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.640903 31006 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.572567 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.642699 32313 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.643435 32313 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.644052 32313 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.643900 32235 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:43.649834 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 77475.8us
I20260809 07:32:43.629981 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.646106 32001 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.632187 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.623432 32265 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.633262 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.646548 32312 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.651031 32249 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.656476 32312 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.656628 32249 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.657508 32249 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.631517 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.632925 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.660185 30899 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.657260 32312 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.652637 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:43.662441 32001 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.653244 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.663583 32234 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:43.668057 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.654754 31006 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.672266 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.675586 30906 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.677695 32277 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.678493 32277 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.678612 32309 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.679255 32277 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
W20260809 07:32:43.676525 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 58760.4us
I20260809 07:32:43.682685 31006 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.682130 32235 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.685573 32265 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.694265 32265 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.695202 32265 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.681561 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.696079 32253 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.699774 32253 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.702057 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.667289 32306 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.700480 32253 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.706470 32306 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.698251 30852 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.712142 32309 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.712872 32309 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.713267 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.713598 32309 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.720113 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.720922 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.726495 30911 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:43.726729 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:43.698125 31006 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.731626 31006 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.733199 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.729275 32306 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.745734 32306 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.749145 32001 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.759394 32001 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.760347 32001 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.764612 32249 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.769057 32249 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:43.776157 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:43.780362 32277 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.737975 30852 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.751070 32263 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:43.784507 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 54437.2us
I20260809 07:32:43.755705 32306 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.755537 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:43.786552 30852 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.790272 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.791507 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.799105 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:43.791744 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 60244.6us
I20260809 07:32:43.752828 32313 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.817772 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.825897 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.808585 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.794399 32230 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.789438 32249 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:43.802729 32234 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:43.824308 32235 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.803153 32304 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.858523 32230 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:32:43.874758 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 83892.2us
I20260809 07:32:43.798648 32312 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.854216 32311 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.894132 30852 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:43.898236 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 80729.3us
I20260809 07:32:43.754963 32309 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.776409 32265 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.904021 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.903330 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:43.903326 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.874574 32235 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:32:43.909781 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 107561us
I20260809 07:32:43.887374 32230 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.882944 32304 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:43.919705 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:43.905704 32311 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:43.909323 32234 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.910518 32235 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.920203 32292 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.930502 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.932250 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.932726 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.934135 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.936269 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.937465 32277 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.939625 32253 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.942935 32316 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:43.944761 32313 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.917166 32304 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.944909 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.945760 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.946636 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.951237 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:43.978924 32230 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.920260 32314 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:43.955467 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:43.963884 32265 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.929932 32263 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.967110 32001 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:43.967165 32309 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:43.971921 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 52346.4us
I20260809 07:32:44.048892 31006 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:43.979377 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47050.3us
I20260809 07:32:43.959861 32312 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:43.978724 32311 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.005738 30909 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:43.979598 32234 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.058625 32234 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:43.981289 32292 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:44.018908 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:44.067966 32292 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.025499 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:43.951877 32318 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:44.042608 30906 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.055430 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:44.072235 30904 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.031729 32314 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.072392 31006 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.079926 32265 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.087810 32235 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.090373 32253 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.091167 32326 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.101138 32314 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.108973 32326 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.110059 32326 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.042250 32230 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:44.106544 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 96607.1us
I20260809 07:32:44.030839 32306 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.074501 32318 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:44.115370 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 90120.7us
I20260809 07:32:44.117317 31005 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.117130 32263 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.115545 30852 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.105703 32265 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:32:44.127102 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 54878.8us
I20260809 07:32:44.129455 31006 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.134078 31006 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.135424 32263 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.136360 32263 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.137270 32263 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.138370 32316 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.139895 32316 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.140141 32327 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.140704 32316 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.141044 30896 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.138631 32253 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.128551 32277 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.144389 32277 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:44.145452 32277 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.118139 32001 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.147574 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.147127 31006 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.149343 32001 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.153537 31006 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.154596 32001 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.130586 32265 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.155297 32001 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.158867 32001 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.156955 31006 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.163162 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.164693 32312 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.168804 31054 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.158435 32313 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.177351 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.190814 31057 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.179234 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:44.179594 32330 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.153452 32327 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.203181 32327 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.188673 32235 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.171465 32312 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.204593 32312 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.206825 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:44.189993 32313 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.210614 32313 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.226577 32306 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.231915 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.242504 31054 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.256438 32001 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.264106 31058 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.272578 31054 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.266453 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.258915 32235 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:44.283322 31045 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.283950 32235 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:44.319259 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 147236us
W20260809 07:32:44.383179 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 123431us
I20260809 07:32:44.398005 31005 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.290096 32292 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.400519 31006 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.289202 32311 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.401285 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.403694 32001 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.408823 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.410046 31058 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.348104 32309 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:44.401273 31057 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:44.428987 32330 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.430042 32330 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.431205 32330 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.434595 30911 raft_consensus.cc:1226] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.439232 31177 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.440181 32311 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.423398 32309 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.446033 32309 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.449661 32292 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.452850 32263 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.420329 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:32:44.419605 31120 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.66:36075 timed out after 1.515s (SENT) [suppressed 5 similar messages]
I20260809 07:32:44.462366 32001 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.434547 32318 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:44.473615 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 53441.6us
I20260809 07:32:44.479758 31058 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:44.483043 30853 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:44.454727 32326 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:44.463706 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 62614.6us
I20260809 07:32:44.480602 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:44.484699 32235 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.473033 32316 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.440266 32277 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.497746 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:44.420217 32304 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.606010 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.464709 32320 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:32:44.595570 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 112204us
W20260809 07:32:44.596190 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 113166us
I20260809 07:32:44.517283 32238 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:44.485410 32288 consensus_queue.cc:1046] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260809 07:32:44.508203 32338 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.607276 32001 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.609948 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:44.613492 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:44.471927 32306 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
I20260809 07:32:44.629520 32271 heartbeater.cc:503] Master 127.28.87.126:41365 was elected leader, sending a full tablet report...
I20260809 07:32:44.598452 32292 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.629899 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:44.600900 32230 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:44.631932 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.633423 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.625576 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:44.640834 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.627800 32319 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.628129 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.623067 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.627470 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:44.609704 32234 consensus_meta.cc:287] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.545s user 0.014s sys 0.014s
I20260809 07:32:44.631389 32331 consensus_queue.cc:1046] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260809 07:32:44.617133 32313 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.643175 30891 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
W20260809 07:32:44.656992 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 237471us
I20260809 07:32:44.653610 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.657948 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.652208 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.623015 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:44.645555 32001 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.660055 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.664974 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.656992 32238 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.665669 32269 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:44.676595 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.671502 31054 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.678403 32001 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.635608 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.681928 31005 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.683391 32001 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.655552 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.685425 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.686178 31172 raft_consensus.cc:1226] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.687768 32304 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.610849 32265 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.689229 32277 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.645676 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.690624 32265 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.680611 32318 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.693004 32326 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.695887 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.699110 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.699712 32001 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.616456 32314 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.700481 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.698956 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.700109 32312 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.713691 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.715281 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.702303 32319 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.694698 32269 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.716791 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.716972 32319 raft_consensus.cc:3032] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:44.717527 32319 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:44.718194 32313 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.704124 32316 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.719731 32269 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.720198 32319 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.702638 31005 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:44.724817 32327 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.726225 32238 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.726651 31049 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.670964 32234 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.730043 31005 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:44.730350 32337 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.730887 32337 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:44.731174 32319 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.731446 32337 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.736052 31059 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.655999 32320 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.745841 32263 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.746085 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.746835 32314 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.733590 32313 raft_consensus.cc:2741] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:44.749406 31045 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:44.749828 32338 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.750360 32316 raft_consensus.cc:2741] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:44.752940 31057 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.757201 31005 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.730301 32234 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.736260 32312 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.768318 32313 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:44.786172 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.787076 31059 raft_consensus.cc:2379] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:44.792963 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.787787 32313 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:44.861544 32327 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.862641 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.858081 32313 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.873342 31049 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.874924 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.877661 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:44.878873 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:44.880970 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:44.888839 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 185488us
I20260809 07:32:44.770418 32330 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.901693 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.903599 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.903779 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.904505 31169 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.905011 31174 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:44.905640 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.903905 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:44.909924 31165 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.912671 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:44.914470 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 299408us
I20260809 07:32:44.915313 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.903618 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:44.903805 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:44.910738 32320 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:44.913636 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:44.921711 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 308367us
I20260809 07:32:44.925848 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:44.927393 30852 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.927033 31006 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.928867 32001 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.931223 31006 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.931928 32001 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.934131 30852 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.936359 30853 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:44.923019 32331 raft_consensus.cc:3032] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:44.933231 32353 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:44.918141 32311 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260809 07:32:44.946640 32311 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:44.947032 32311 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:44.948035 32311 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.949661 32311 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:44.950448 32353 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:44.938059 32292 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:44.951996 32292 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:44.952617 30853 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:44.952518 30852 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:44.957113 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.958604 31006 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.959884 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:44.949780 32313 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.908660 32332 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260809 07:32:44.941108 32288 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.956902 32001 consensus_queue.cc:1046] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:44.953238 32346 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:44.969028 31058 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.936828 32320 consensus_queue.cc:1046] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260809 07:32:44.971763 32235 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:44.972429 32235 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.961360 32303 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:44.970037 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:44.964988 32312 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.952914 32331 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:44.979192 32303 raft_consensus.cc:686] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:44.979017 32350 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260809 07:32:44.958797 32316 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.976917 32001 raft_consensus.cc:3032] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:44.982743 32001 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:44.951802 32353 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.925473 32234 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 07:32:44.975905 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:44.981967 31048 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.964216 32338 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:44.990931 32303 consensus_queue.cc:237] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:44.997511 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:44.991467 32269 raft_consensus.cc:3032] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:44.992197 31049 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.981838 32309 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:44.995545 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:45.006896 31059 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.007301 31058 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.007529 31049 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.008185 31051 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.994055 32330 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:44.998745 32288 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.000074 32277 raft_consensus.cc:3032] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.012399 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:45.009225 31049 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.975142 32235 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.002737 31045 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.003189 31055 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.003435 32321 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.003911 32316 consensus_queue.cc:1046] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:45.007090 32309 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.009845 31058 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:44.996456 32001 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.011061 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.011502 31051 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.011693 31048 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.011855 32332 raft_consensus.cc:3032] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.003825 32292 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.012486 31046 raft_consensus.cc:1226] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.001745 32306 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:45.000607 30853 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.012723 32277 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:32:45.016594 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 87851us
I20260809 07:32:44.979522 32331 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.034377 32331 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.020252 32001 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.004478 32346 raft_consensus.cc:686] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.034696 32306 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:45.035310 32306 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.004853 32269 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:45.030645 32292 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.030936 32349 raft_consensus.cc:3032] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.021669 32332 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:44.975423 32348 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260809 07:32:45.040521 30853 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.045445 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.035718 32346 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.034188 32277 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.052798 32277 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.053387 32345 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.036695 32288 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.058946 30853 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.059126 32304 raft_consensus.cc:3032] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.059662 32304 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.060299 32345 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.061389 32340 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.061007 32304 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.061939 32340 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.050050 32321 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.060109 30853 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.043910 32332 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.066553 32320 consensus_queue.cc:1046] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:45.061049 32235 consensus_queue.cc:1046] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 07:32:45.043972 32269 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.068149 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.051837 32349 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:45.087610 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.091163 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.097249 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.071225 32288 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.067690 32330 raft_consensus.cc:3032] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.109064 32352 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.111971 30853 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.118468 32332 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.089030 32321 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.123319 30853 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.124388 30853 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.102092 32269 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.103147 32348 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.130868 32350 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.107623 32313 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:45.129006 32288 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.110491 32345 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.133100 32319 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:45.066699 32340 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.110805 32304 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.147012 32337 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.111037 32330 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:45.145953 32238 raft_consensus.cc:3032] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.209059 32238 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:45.210584 32238 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.210873 32330 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.212708 32238 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.213132 32330 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.220415 31169 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.222627 32349 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.229650 32349 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.236995 32306 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.239989 32337 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:45.240628 31054 raft_consensus.cc:1226] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.250528 32340 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.248119 32309 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.305096 32306 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:45.334762 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 243122us
I20260809 07:32:45.345050 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.351408 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.353493 32269 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.351073 32238 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.353200 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.363157 32234 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.366405 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.374504 32238 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.377167 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.378491 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.379874 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.385150 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.385296 32238 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.387058 32238 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.388252 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.388952 32238 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.389889 32269 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.385121 32234 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.392588 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.393949 32269 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.409662 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.419191 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.419309 32238 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.419991 32357 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.429900 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.432310 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.433043 32234 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.433977 32234 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.435638 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.436463 32234 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.450003 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.452126 32311 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.462270 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.465337 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.465994 30909 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:45.464339 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:45.468532 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.469107 30911 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:45.469962 31054 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.471586 31006 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:45.472440 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.476714 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.477027 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.478125 31054 raft_consensus.cc:2379] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.469473 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.469359 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.482122 30896 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.517671 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.484334 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.504459 32238 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.482862 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.530040 32238 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.502357 32269 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.531541 30891 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:45.532035 32269 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.476423 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.482993 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.535465 32269 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.475509 32311 consensus_queue.cc:1046] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260809 07:32:45.494877 32357 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.537012 32311 raft_consensus.cc:3032] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.537614 32311 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:45.538928 32357 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:45.543828 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 417838us
I20260809 07:32:45.545053 30853 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.478618 32330 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.471836 32234 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260809 07:32:45.531049 30900 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:45.551249 30853 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.561846 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.568869 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.570127 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.553603 32340 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.577464 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.556417 32330 raft_consensus.cc:686] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:32:45.586165 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.603s user 0.009s sys 0.044s
W20260809 07:32:45.586467 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.604s user 0.009s sys 0.044s
I20260809 07:32:45.570801 32304 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.576159 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.591235 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.594201 32238 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.594938 32238 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.595693 32238 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.598536 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:45.598629 32238 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.601518 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.602094 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.602619 31005 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.603794 32316 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.603936 32320 raft_consensus.cc:3032] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.604570 32320 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:45.604902 31005 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.606001 32316 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.606997 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.582857 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.611384 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.575392 32303 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260809 07:32:45.583580 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.583664 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.626384 31165 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.625849 32235 raft_consensus.cc:3032] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.627650 32235 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:45.629211 31164 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:45.628947 32235 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.630084 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.594787 32330 consensus_queue.cc:237] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.631424 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.632061 30904 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:45.632948 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.606041 32320 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.613065 32304 consensus_queue.cc:1046] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:45.636008 32320 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.613462 32369 consensus_queue.cc:1046] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260809 07:32:45.636387 31165 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.616201 32316 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.615491 32303 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.638839 32303 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.641351 30909 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.590956 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.644106 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.640151 32303 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.587823 32311 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.647226 32303 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.649873 32357 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.615003 31182 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.616606 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.655802 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.656152 32235 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.659346 31046 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:45.660645 30898 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.619527 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.620677 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.621842 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.622736 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.623075 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.624315 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.608937 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:45.612522 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.672389 32316 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260809 07:32:45.673219 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 60770.2us
I20260809 07:32:45.665927 32311 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.671412 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:45.676823 32269 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.687973 32269 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.691201 32269 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.697268 32269 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.685515 30896 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.692741 30899 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.699594 32269 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.703215 31181 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.694530 31170 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.695770 32366 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.686782 30906 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.704367 32366 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.707477 32366 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.696444 32234 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:45.708833 32269 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.708379 32234 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.697405 30900 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
W20260809 07:32:45.710620 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 98363.6us
I20260809 07:32:45.710803 32230 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:45.711434 32234 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:45.712581 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 101654us
I20260809 07:32:45.713287 32277 consensus_queue.cc:1046] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:45.719182 32353 raft_consensus.cc:3032] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.719630 32353 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.720692 32353 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.721858 32345 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.722370 32345 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.722537 32353 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.723244 31051 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.724854 30904 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.725426 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.726136 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.727975 30904 raft_consensus.cc:2379] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.730772 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.731498 31058 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.731528 31120 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:45.730418 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.733382 31055 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.733414 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.736936 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.754671 31054 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.734624 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:45.752496 32001 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:45.718153 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.754586 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.755527 30853 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.756161 32001 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.751057 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.760128 32001 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:45.761927 30896 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.762957 32304 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.763415 32304 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.755753 32277 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260809 07:32:45.764254 32304 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.736002 32366 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:45.714478 32292 raft_consensus.cc:3032] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.756637 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:45.768332 32370 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:45.773541 32370 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:45.774012 31059 raft_consensus.cc:2379] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.774113 32370 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:45.775146 32331 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.775607 32331 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.777304 31181 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:45.769464 32306 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:45.769589 32369 raft_consensus.cc:3032] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.778748 32369 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.770008 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:45.735463 32346 consensus_queue.cc:1046] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260809 07:32:45.770530 31045 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:45.772297 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.779912 32369 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.723527 32345 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.783094 32369 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.771540 31172 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.780570 32306 raft_consensus.cc:3032] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.781744 32346 raft_consensus.cc:3032] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.776759 32331 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.784490 32265 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.786826 32346 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.793105 32230 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 07:32:45.795840 32331 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.792351 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:45.783025 31005 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.798907 30904 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:45.799891 31005 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.800549 32316 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.773317 32292 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.801676 32313 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.802281 32313 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:45.803256 32292 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.795785 32346 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.804195 32313 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.804857 32292 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.805362 32346 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.772862 32366 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:45.806643 32366 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.808421 31005 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.815966 32330 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.816421 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.817155 32001 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.785856 32306 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.826010 32306 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.827399 32306 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.830430 31172 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.831027 31177 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.808809 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:45.836684 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:45.837500 32292 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:45.785295 32345 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.838630 32230 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.839752 32230 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.835839 30853 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.840766 32292 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.840575 32230 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.841326 32292 raft_consensus.cc:686] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.841881 32230 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.842334 32292 consensus_queue.cc:237] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.822247 31174 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:45.851982 32277 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:45.852288 32343 consensus_queue.cc:1046] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:45.853166 32277 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.853526 32350 raft_consensus.cc:3032] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.853555 32277 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.855605 32277 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.837489 32303 consensus_queue.cc:1046] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260809 07:32:45.856917 32277 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.857205 32343 raft_consensus.cc:3032] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:45.857558 32343 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.858408 32343 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.861765 32343 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.853951 32350 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.846107 32235 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:45.862732 31005 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.865103 32235 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:45.865414 32330 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.861912 31169 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:45.848565 32311 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:45.866262 32330 raft_consensus.cc:686] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:45.867543 32235 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.869135 32330 consensus_queue.cc:237] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.865737 32350 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.871171 32350 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.874939 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.876243 30853 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.876704 32369 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.866662 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.873049 32303 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:32:45.882165 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.884141 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.884246 32361 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.884953 32332 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.890744 32369 raft_consensus.cc:686] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.885068 32361 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.894908 31005 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.872880 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.868376 32311 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:45.895998 32001 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.897217 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.898049 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.902953 31005 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.903105 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.904585 32001 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.905092 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.905882 32001 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.906209 31005 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.906296 32001 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:45.908366 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:45.910398 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.894119 32292 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.911538 31005 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:45.907152 32001 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.866544 32370 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.914795 31006 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:45.900208 32311 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.897879 32303 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.921172 32292 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.892590 32369 consensus_queue.cc:237] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.908255 32357 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.915604 32253 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.915604 32359 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.930953 31045 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:45.912847 32238 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:45.920055 32316 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.909771 32265 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:45.909283 32313 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:45.940040 30744 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:45.943578 32366 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.944796 32370 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:45.945492 31177 raft_consensus.cc:1226] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:45.964694 30744 catalog_manager.cc:5495] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:45.973788 30744 catalog_manager.cc:5495] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:45.949884 32303 raft_consensus.cc:686] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:45.952791 32253 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:45.993700 31177 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:45.990581 32253 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:45.985138 32313 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:46.022578 31177 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:45.991750 31059 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:45.988330 32292 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.038902 32321 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:46.015102 32346 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
W20260809 07:32:46.117662 32380 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.057s user 0.000s sys 0.002s
I20260809 07:32:46.062608 32303 consensus_queue.cc:237] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.143360 32332 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.115193 32366 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:46.159289 32311 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.172729 32235 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.196451 32235 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:46.197890 32311 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:46.205976 32238 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.209770 32357 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.224380 32359 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.316665 31172 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:46.337291 32350 consensus_queue.cc:1046] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260809 07:32:46.344492 32331 raft_consensus.cc:3032] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:46.345108 32331 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:46.346786 32331 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.412657 32331 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.423152 31174 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.456554 32350 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260809 07:32:46.520208 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.601s user 0.014s sys 0.015s
W20260809 07:32:46.532099 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.615s user 0.015s sys 0.015s
W20260809 07:32:46.534719 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.549s user 0.014s sys 0.026s
W20260809 07:32:46.545620 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.574s user 0.014s sys 0.028s
W20260809 07:32:46.558609 32265 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.546s user 0.006s sys 0.011s
W20260809 07:32:46.561818 32265 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.582s user 0.007s sys 0.012s
W20260809 07:32:46.566164 31057 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s user 0.000s sys 0.043s
W20260809 07:32:46.566877 31057 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s user 0.000s sys 0.044s
W20260809 07:32:46.554222 32001 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s user 0.003s sys 0.022s
W20260809 07:32:46.553853 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 634790us
W20260809 07:32:46.574067 32001 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s user 0.004s sys 0.022s
W20260809 07:32:46.574337 31006 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.607s user 0.015s sys 0.015s
W20260809 07:32:46.574465 30744 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s user 0.020s sys 0.010s
W20260809 07:32:46.580377 32253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s user 0.011s sys 0.010s
W20260809 07:32:46.580895 32253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s user 0.011s sys 0.010s
I20260809 07:32:46.565356 32265 raft_consensus.cc:686] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:46.583459 32265 consensus_queue.cc:237] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:46.586400 31003 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.561s user 0.010s sys 0.040s
W20260809 07:32:46.586872 31003 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.609s user 0.011s sys 0.040s
I20260809 07:32:46.591692 32001 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.592489 32001 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:46.596704 32313 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.509s user 0.016s sys 0.011s
W20260809 07:32:46.597169 32313 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.557s user 0.016s sys 0.011s
I20260809 07:32:46.597509 32313 raft_consensus.cc:686] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:46.599596 32387 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.600546 32387 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:46.601672 31006 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.684s user 0.015s sys 0.016s
I20260809 07:32:46.605294 32387 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:46.602653 30744 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.615s user 0.020s sys 0.010s
I20260809 07:32:46.612238 32001 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:46.613615 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 698856us
W20260809 07:32:46.614306 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.621s user 0.016s sys 0.019s
W20260809 07:32:46.616616 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.700s user 0.017s sys 0.019s
I20260809 07:32:46.599292 32313 consensus_queue.cc:237] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.627035 31177 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.628938 32303 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260809 07:32:46.619222 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 705587us
I20260809 07:32:46.636794 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:46.640774 32269 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.641331 31164 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:46.646981 31181 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:46.652649 31120 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:46.656132 32397 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.631390 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:46.664237 31059 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.665588 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:46.654856 32316 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.589s user 0.011s sys 0.026s
I20260809 07:32:46.670207 31057 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:46.670284 31170 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 07:32:46.672578 32316 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.692s user 0.011s sys 0.026s
I20260809 07:32:46.672969 32316 raft_consensus.cc:686] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:46.674117 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.674340 32316 consensus_queue.cc:237] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.675153 30904 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:46.676632 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.677613 30899 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:46.678934 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.680037 30896 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:46.658587 32269 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 07:32:46.686496 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 55362us
I20260809 07:32:46.686729 32313 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:46.687178 32313 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.687628 32313 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.687430 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:46.711612 31177 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.711042 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:46.719208 32398 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:46.720027 32398 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.723266 31049 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.724550 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:46.728996 32397 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.722229 32398 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.730599 32387 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.731158 32387 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.733055 32387 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.736387 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.738070 30891 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.738619 32292 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:46.738924 30906 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.740988 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.741075 32406 consensus_queue.cc:1046] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:46.742758 32406 consensus_queue.cc:1046] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:46.759325 31045 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.761905 31059 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.767267 32401 raft_consensus.cc:3032] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:46.767936 32401 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:46.769306 32401 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.778299 32409 raft_consensus.cc:3032] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:46.778762 32409 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:46.780125 32409 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.781697 32409 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:46.770660 32408 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.783174 32408 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.785434 32408 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.757792 32292 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.786424 32292 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.788892 32292 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.790096 32369 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:46.790735 32369 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.792946 32369 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.796849 31057 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.799372 30745 catalog_manager.cc:5495] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:46.805308 31057 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.808010 32304 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:46.814850 32331 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:46.816352 32331 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:46.770040 32427 consensus_queue.cc:1046] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260809 07:32:46.838352 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:46.840577 31177 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.842272 32001 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:46.842890 32001 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.845149 32001 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.847524 32398 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:46.848049 32398 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.849895 32398 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.862982 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:46.868184 31177 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:46.832542 32427 raft_consensus.cc:3032] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:46.871496 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:46.878548 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:46.878830 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:46.879726 31170 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:46.880156 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:46.880921 32398 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.881073 31057 raft_consensus.cc:2379] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:46.882037 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:46.882598 31181 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.871203 31174 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.870985 32427 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:46.887511 32253 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:46.891490 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:46.894063 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 84075.5us
I20260809 07:32:46.885527 31049 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.896214 32265 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 07:32:46.898967 31046 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.898694 32427 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.898725 32369 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 07:32:46.905946 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:46.906872 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:46.911440 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 48580.2us
I20260809 07:32:46.911880 32313 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.913259 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:46.916661 32401 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:46.919817 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:46.923296 32442 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260809 07:32:46.930583 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:46.937755 32265 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:46.938824 31059 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:46.935081 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:46.931178 32387 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:46.947654 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:46.950497 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.952538 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.954270 32319 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 07:32:46.952662 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.955248 32409 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:46.952858 31181 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:46.956075 32387 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.955756 32409 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.955745 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:46.961822 32384 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:46.962425 32384 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:46.938666 31169 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:46.964558 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.960515 32319 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:32:46.959663 32397 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:46.953307 30899 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:46.967475 32397 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:46.979506 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:46.983206 30904 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:46.965288 32409 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:46.967605 32384 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:46.980248 32397 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.033663 31054 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:47.050761 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:47.050793 30899 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.050472 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:47.035729 32427 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
W20260809 07:32:47.070962 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 105993us
I20260809 07:32:47.065378 32265 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.073982 32313 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.087728 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.071029 32387 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.092819 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:47.095964 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:47.096625 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:47.106181 31049 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:47.107070 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.107530 32402 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.108042 32402 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.109326 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.111074 30853 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:47.112761 32452 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.116729 32452 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:47.116849 30853 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:47.118557 32452 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.092324 32269 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.124506 31181 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.125442 32384 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:47.125591 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.126403 31170 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:47.125833 32384 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:47.127192 32384 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.127040 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.127650 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.091795 32265 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.134109 31048 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:47.135807 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.138334 31005 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.130050 32402 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.151798 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.151013 32398 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:47.153731 32398 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:47.154091 32398 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.154404 32343 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.154748 32343 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.154888 32292 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:47.155527 32292 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.155095 32343 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.157196 32461 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.157649 32292 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.133319 32269 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.136873 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:47.152092 32001 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:47.159410 32402 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.167793 31006 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:47.171425 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52409us
I20260809 07:32:47.212886 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.174928 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:47.215435 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 49638.7us
W20260809 07:32:47.215858 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 48152.5us
I20260809 07:32:47.215730 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.222378 31177 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:47.175247 32401 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.212548 32253 consensus_queue.cc:1046] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260809 07:32:47.223721 32406 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.225682 32452 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.157667 32461 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.223495 32292 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:47.213889 32316 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260809 07:32:47.226904 32401 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.227617 32313 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260809 07:32:47.228617 32402 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.249243 32401 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.248677 32369 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:47.286641 32468 log.cc:924] Time spent T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.111s user 0.000s sys 0.001s
I20260809 07:32:47.288533 31181 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.289497 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:47.291186 32465 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260809 07:32:47.291438 32461 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.254945 32452 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.280633 32265 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.274358 32292 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.281421 32369 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.321624 31054 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.325089 32369 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.307216 32269 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.336683 32402 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.341813 31045 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.341691 32397 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.345398 32397 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.347807 32343 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.315078 32401 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.354974 32452 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.355835 32401 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:47.365983 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 148447us
I20260809 07:32:47.366793 32387 raft_consensus.cc:3032] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:47.378897 32387 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:47.380640 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40564 (request call id 146) took 1473 ms. Trace:
I20260809 07:32:47.375407 32292 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.380486 32387 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.383976 32387 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.381050 30744 rpcz_store.cc:276] 0809 07:32:45.922728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:45.929171 (+ 6443us) service_pool.cc:227] Handling call
0809 07:32:47.380527 (+1451356us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:32:46.563342 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:32:46.564314 (+ 972us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:32:46.564386 (+ 72us) write_op.cc:435] Acquired schema lock
0809 07:32:46.564425 (+ 39us) tablet.cc:660] Decoding operations
0809 07:32:46.565506 (+ 1081us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:32:46.565624 (+ 118us) write_op.cc:641] Partition lock acquired for write op
0809 07:32:46.565681 (+ 57us) tablet.cc:683] Acquiring locks for 3 operations
0809 07:32:46.566270 (+ 589us) tablet.cc:699] Row locks acquired
0809 07:32:46.566304 (+ 34us) write_op.cc:260] PREPARE: finished
0809 07:32:46.602178 (+ 35874us) write_op.cc:270] Start()
0809 07:32:46.616533 (+ 14355us) write_op.cc:276] Timestamp: P: 1786260766578144 usec, L: 0
0809 07:32:46.616590 (+ 57us) op_driver.cc:331] REPLICATION: starting
0809 07:32:46.642670 (+ 26080us) log.cc:842] Serialized 1303 byte log entry
0809 07:32:46.887896 (+245226us) op_driver.cc:444] REPLICATION: finished
0809 07:32:46.972147 (+ 84251us) write_op.cc:301] APPLY: starting
0809 07:32:46.972240 (+ 93us) tablet.cc:1365] starting BulkCheckPresence
0809 07:32:46.972391 (+ 151us) tablet.cc:1368] finished BulkCheckPresence
0809 07:32:46.972527 (+ 136us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:32:46.973314 (+ 787us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:32:46.973615 (+ 301us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0809 07:32:46.973678 (+ 63us) write_op.cc:312] APPLY: finished
0809 07:32:46.974180 (+ 502us) log.cc:842] Serialized 34 byte log entry
0809 07:32:46.974421 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:46.974533 (+ 112us) write_op.cc:454] Released schema lock
0809 07:32:46.974594 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0809 07:32:46.976635 (+ 2041us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:46.976665 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":227559,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":83923,"apply.run_cpu_time_us":3154,"apply.run_wall_time_us":4835,"num_ops":3,"prepare.queue_time_us":575146,"prepare.run_cpu_time_us":15558,"prepare.run_wall_time_us":142417,"raft.queue_time_us":123020,"raft.run_cpu_time_us":31564,"raft.run_wall_time_us":134311,"replication_time_us":271124,"thread_start_us":825035,"threads_started":4,"wal-append.queue_time_us":70757}]]}
I20260809 07:32:47.394991 32384 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.387840 31006 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.401736 32384 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.401780 32401 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:47.351613 32343 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.404578 31006 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.405562 32313 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.411389 32384 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:47.413400 32462 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.292s user 0.000s sys 0.005s
I20260809 07:32:47.405949 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.421828 32313 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.422566 32313 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.423120 32313 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.412225 32384 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.429003 32384 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.430663 32398 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:47.446405 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 221118us
I20260809 07:32:47.445844 31006 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:47.452310 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 227226us
I20260809 07:32:47.454360 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.405385 32401 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.457950 32316 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:47.462469 32461 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:47.463984 32461 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.465520 32353 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.466219 32353 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.467774 32461 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.457504 32265 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260809 07:32:47.469790 32353 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.469908 32398 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.463042 32313 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.476075 31165 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.476608 32313 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.477164 31181 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.478978 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.479169 31174 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.479595 31182 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:47.500494 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.502910 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:47.506608 31182 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.506807 31054 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:47.509449 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:47.510542 31165 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:47.512130 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.513866 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.517251 32253 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:47.517513 32483 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:47.518181 32483 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.518572 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.518873 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:47.519372 31049 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:47.519466 31177 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.500670 32353 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:47.522688 32408 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.525300 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:47.527311 31181 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.526780 31058 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.509245 32292 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.531013 30904 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.525611 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.525723 32319 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.527244 32408 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.564441 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:47.520514 32401 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260809 07:32:47.531441 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.568873 31045 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:47.570025 31046 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:47.568120 31177 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:47.534516 32476 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.566790 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.565275 32408 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.558707 32331 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.581461 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.588063 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:47.588874 30911 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.588543 32331 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.559331 32292 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.597822 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:47.579941 32452 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.623014 30899 raft_consensus.cc:1261] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.573977 32353 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:47.634645 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 44463.3us
I20260809 07:32:47.590246 32476 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260809 07:32:47.637312 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 45359.5us
I20260809 07:32:47.633782 32265 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:47.583509 32483 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.576372 32319 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.621012 32398 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:47.622494 32303 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.588585 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.622429 30906 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.650113 32477 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.658744 31177 raft_consensus.cc:1261] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.664932 30911 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:47.665861 31182 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:47.665836 32303 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.673231 30904 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.667989 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.668720 30890 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:47.673758 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.674448 32001 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:47.692256 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:47.693032 31054 raft_consensus.cc:2379] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:47.695046 32001 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:47.695506 32001 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:47.672524 32265 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.697089 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.670663 32398 raft_consensus.cc:686] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:47.700259 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:47.700030 32477 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.697630 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.690411 32303 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.708081 30898 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:47.664412 32409 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260809 07:32:47.712250 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:47.672817 32427 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.700768 32398 consensus_queue.cc:237] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.714126 32475 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.715917 32265 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.716440 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.668744 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:47.718031 32001 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.704167 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.725317 31057 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:47.725023 32475 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.726658 32477 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.727094 32343 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.728704 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.706080 32331 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.750187 32475 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:47.752969 32343 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.756319 32343 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.759321 32490 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.718744 31059 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.761018 32488 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.761623 32488 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.764066 32488 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.765790 32353 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.759923 32490 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.780710 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:47.781312 31174 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:47.782442 31006 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.783139 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.768501 32490 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.723656 32427 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.786398 32303 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.774287 32292 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.797022 32269 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:47.801940 32269 raft_consensus.cc:686] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:47.791564 32488 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.815474 32408 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.819732 31058 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:47.815950 30898 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.821974 32319 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.802214 32253 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.828892 31048 raft_consensus.cc:2354] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:47.824349 32408 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.819638 32488 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:32:47.833165 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 257355us
I20260809 07:32:47.831336 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:47.838958 31174 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.839641 30852 leader_election.cc:398] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:47.839951 30852 leader_election.cc:401] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:32:47.840386 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.840715 32331 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:47.841126 30904 raft_consensus.cc:2379] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:47.841122 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.842031 32292 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.842417 32292 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.842773 32292 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.833814 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:47.844022 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.844626 30904 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.844792 32475 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.845764 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.848845 30904 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:32:47.850045 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:47.851262 32292 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.852357 32408 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:47.853181 31174 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:47.854167 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:47.855363 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.858165 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.858645 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.861438 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.862687 30911 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.824554 31051 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.873020 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.882984 30904 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:47.823767 32269 consensus_queue.cc:237] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.859467 32488 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.892478 30906 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:47.888178 31120 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:47.896175 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.897182 31058 raft_consensus.cc:2379] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:47.883798 32401 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:47.909628 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.830412 32253 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.916069 32481 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:47.916345 32409 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:47.918978 32491 consensus_queue.cc:1046] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:47.920109 32491 raft_consensus.cc:3032] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:47.920512 32491 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:47.924793 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.895541 32461 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:47.898126 32331 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:47.930927 32389 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.931336 32389 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.931732 32389 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.932116 32331 raft_consensus.cc:686] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:47.933120 32331 consensus_queue.cc:237] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.925869 32401 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:47.928179 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:47.921299 32491 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:47.935599 32401 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.939980 32475 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:47.940428 32475 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:47.940743 32475 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:47.911072 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:47.944796 30899 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:47.890799 32408 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:47.927652 32343 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:48.106048 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:48.107529 31048 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:48.105401 32485 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:47.985769 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.108719 32485 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.056655 32303 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.110555 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.948069 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:47.956065 30906 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:47.952756 32387 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
I20260809 07:32:48.133272 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:48.070721 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.111723 32491 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:48.139508 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:48.140133 30906 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:48.134085 30744 catalog_manager.cc:5495] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:32:48.144579 31054 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
W20260809 07:32:48.131577 32304 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.014s sys 0.026s
W20260809 07:32:48.057283 31170 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s user 0.010s sys 0.020s
I20260809 07:32:48.036041 30891 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:48.127797 32397 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:47.951236 32427 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:47.952291 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:48.175609 31165 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s user 0.008s sys 0.010s
I20260809 07:32:48.186606 30744 catalog_manager.cc:5495] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 2, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
W20260809 07:32:48.202623 31165 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s user 0.009s sys 0.010s
W20260809 07:32:48.189889 31164 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s user 0.055s sys 0.036s
W20260809 07:32:48.188094 32316 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s user 0.013s sys 0.045s
W20260809 07:32:48.148643 32304 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s user 0.014s sys 0.027s
W20260809 07:32:48.167109 31170 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s user 0.012s sys 0.020s
I20260809 07:32:47.945642 32265 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.206s
I20260809 07:32:48.216574 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:48.219450 30906 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:48.150918 32253 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:48.216536 30909 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s user 0.000s sys 0.058s
W20260809 07:32:48.228397 30909 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.577s user 0.000s sys 0.058s
I20260809 07:32:48.131218 32343 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.216317 31164 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s user 0.058s sys 0.036s
W20260809 07:32:48.216681 32316 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s user 0.013s sys 0.045s
W20260809 07:32:48.240316 31182 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s user 0.017s sys 0.044s
W20260809 07:32:48.248708 31182 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s user 0.017s sys 0.044s
I20260809 07:32:48.191483 31045 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:48.263302 30744 catalog_manager.cc:5495] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:32:48.218976 31172 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:48.253072 32481 raft_consensus.cc:686] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:48.255368 32409 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:48.258024 32485 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:48.298602 32461 raft_consensus.cc:686] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:48.329097 32384 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:48.329641 32384 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.331781 31048 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:48.287228 32490 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.316031 30744 catalog_manager.cc:5495] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
W20260809 07:32:48.310083 31118 thread.cc:636] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.539s user 0.000s sys 0.032s
W20260809 07:32:48.336695 31118 thread.cc:602] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.741s user 0.001s sys 0.032s
W20260809 07:32:48.334408 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 222821us
I20260809 07:32:48.319167 32461 consensus_queue.cc:237] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.320710 32409 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.515941 32384 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:48.519381 32485 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:48.519959 32485 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.520370 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.521114 31059 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
W20260809 07:32:48.508085 32525 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.213s user 0.002s sys 0.002s
I20260809 07:32:48.523021 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 416) took 1023 ms. Trace:
I20260809 07:32:48.523502 31170 rpcz_store.cc:276] 0809 07:32:47.499619 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.500595 (+ 976us) service_pool.cc:227] Handling call
0809 07:32:47.518624 (+ 18029us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.519036 (+ 412us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.519066 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.519519 (+ 453us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.237488 (+717969us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.242648 (+ 5160us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.251373 (+ 8725us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:48.522787 (+271414us) raft_consensus.cc:1671] finished
0809 07:32:48.522815 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:48.522960 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6751,"raft.queue_time_us":2873,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":9134,"thread_start_us":696755,"threads_started":1,"wal-append.queue_time_us":734951}
I20260809 07:32:48.394488 32408 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.363826 32490 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.580421 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 217580us
I20260809 07:32:48.370636 32387 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.167s
W20260809 07:32:48.621587 31174 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.642s user 0.000s sys 0.007s
W20260809 07:32:48.622097 31174 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.643s user 0.000s sys 0.007s
I20260809 07:32:48.632251 32001 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:32:48.633407 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52114.2us
I20260809 07:32:48.634307 31005 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:48.636598 31005 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:48.632946 32001 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.635138 32331 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s user 0.004s sys 0.004s
W20260809 07:32:48.654951 32331 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s user 0.005s sys 0.005s
I20260809 07:32:48.658986 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:48.659705 31051 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:48.670024 32253 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:48.670665 32253 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:48.671734 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 395) took 1087 ms. Trace:
I20260809 07:32:48.671093 32253 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:48.672097 30909 rpcz_store.cc:276] 0809 07:32:47.583734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.586647 (+ 2913us) service_pool.cc:227] Handling call
0809 07:32:47.621172 (+ 34525us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:47.621376 (+ 204us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:32:48.239740 (+618364us) log.cc:842] Serialized 14 byte log entry
0809 07:32:48.666486 (+426746us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:48.666533 (+ 47us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:48.666571 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.666630 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:48.666812 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":239953,"prepare.queue_time_us":20580,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":240867,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":763985,"threads_started":2}
I20260809 07:32:48.665266 32485 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:48.698510 32001 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.683235 32534 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:48.698118 32505 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.672648 32452 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.672545 32316 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:48.672403 32398 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.536374 32481 consensus_queue.cc:237] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.704928 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:48.707819 32316 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:48.709795 31174 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
W20260809 07:32:48.701833 32488 consensus_meta.cc:287] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.779s user 0.005s sys 0.004s
I20260809 07:32:48.712527 32510 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:48.699973 32534 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.711221 32398 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.714798 32540 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.702466 32265 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:32:48.724177 32531 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.098s user 0.008s sys 0.008s
I20260809 07:32:48.724493 31175 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:48.724948 32398 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.725251 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:48.725374 32483 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.725677 32534 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:48.721066 32545 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:48.705518 32387 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.706955 32331 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.735219 32536 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:48.707995 32505 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.747514 31006 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:48.732064 32353 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:48.713717 32452 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.749226 30890 consensus_meta.cc:287] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.858s user 0.008s sys 0.003s
I20260809 07:32:48.713642 32343 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.754771 32387 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.750370 32483 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.756763 32545 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.762378 32001 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:48.922513 32001 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:48.922996 32001 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:48.750646 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:48.903378 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.864s user 0.011s sys 0.022s
I20260809 07:32:48.755728 32350 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:48.735150 32540 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.775435 32490 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.721490 32488 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.738896 32265 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.713083 32316 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.892956 31118 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.543s user 0.018s sys 0.030s
I20260809 07:32:48.893424 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:48.773121 32253 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.755857 32331 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.911921 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 229273us
I20260809 07:32:48.927841 32343 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:32:48.775642 32491 consensus_meta.cc:287] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.627s user 0.007s sys 0.000s
W20260809 07:32:48.928911 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.942s user 0.011s sys 0.022s
I20260809 07:32:48.783391 32505 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:48.898212 32269 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s user 0.097s sys 0.113s
I20260809 07:32:48.936954 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:48.951532 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.801203 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.868096 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:48.894774 30890 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
W20260809 07:32:48.899540 31121 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.784s user 0.006s sys 0.040s
I20260809 07:32:48.737612 32510 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.940773 31118 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.591s user 0.018s sys 0.030s
I20260809 07:32:48.866725 32408 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260809 07:32:48.771647 32353 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.799032 32475 consensus_meta.cc:287] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.858s user 0.007s sys 0.003s
I20260809 07:32:48.940773 30900 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:48.943867 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:48.946220 31055 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:48.772006 32536 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.938951 32559 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:48.952180 31048 raft_consensus.cc:2354] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:48.939229 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:48.957185 32389 consensus_meta.cc:287] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.025s user 0.009s sys 0.004s
W20260809 07:32:48.961635 31121 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.893s user 0.006s sys 0.041s
I20260809 07:32:48.942966 32556 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.937333 32350 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:48.951396 32269 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s user 0.097s sys 0.113s
I20260809 07:32:48.956851 32554 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.951701 30909 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:48.941498 31170 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:48.983829 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.972499 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:48.957732 32461 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:48.963097 32331 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.946121 32490 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:48.958987 32488 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.962785 32265 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:48.960028 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:48.947738 32343 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.982383 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:49.047956 30900 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:48.960546 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 577) took 1108 ms. Trace:
I20260809 07:32:48.960057 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:49.012179 31120 leader_election.cc:398] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:48.975590 30891 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:49.013695 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:48.969290 32475 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.983939 32556 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:48.972986 32566 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:49.008697 32481 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.056403 30909 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:49.049057 31120 leader_election.cc:401] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260809 07:32:49.063409 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53954.1us
I20260809 07:32:49.013083 32505 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:32:49.126787 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 114741us
I20260809 07:32:48.976572 32559 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.057158 32269 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.135350 30900 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:48.974457 32389 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.088106 32265 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.015937 32554 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.127465 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:49.089618 32387 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.121331 32483 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.015199 32461 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.043918 32490 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:49.504910 32564 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.431s user 0.000s sys 0.000s
W20260809 07:32:49.444947 32001 consensus_meta.cc:287] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.508s user 0.011s sys 0.000s
I20260809 07:32:49.443935 32475 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:49.125468 32505 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:32:49.424021 32570 log.cc:924] Time spent T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.101s user 0.000s sys 0.000s
W20260809 07:32:49.290340 32551 log.cc:924] Time spent T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.210s user 0.002s sys 0.000s
I20260809 07:32:49.643162 32546 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.302421 32536 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:49.120425 32486 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.330798 32459 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.650633 32566 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:49.444453 32556 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.147274 32452 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.653966 32475 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:49.318521 31174 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260809 07:32:49.505194 32526 log.cc:924] Time spent T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.130s user 0.000s sys 0.011s
W20260809 07:32:49.334008 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 284939us
W20260809 07:32:49.504156 32530 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.961s user 0.005s sys 0.007s
I20260809 07:32:49.642979 32001 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:49.330760 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 63583us
W20260809 07:32:49.316766 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 193574us
W20260809 07:32:49.386761 32569 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.260s user 0.000s sys 0.001s
I20260809 07:32:49.277135 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 564) took 1360 ms. Trace:
I20260809 07:32:49.662034 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 575) took 1628 ms. Trace:
W20260809 07:32:49.643447 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 516035us
W20260809 07:32:49.504618 32523 log.cc:924] Time spent T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.843s user 0.008s sys 0.002s
I20260809 07:32:49.652475 32265 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:49.226994 31051 consensus_meta.cc:287] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.567s user 0.009s sys 0.002s
I20260809 07:32:49.338101 32253 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.632038 32491 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.288538 30891 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:49.666433 32547 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.566334 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 580) took 1686 ms. Trace:
I20260809 07:32:49.387282 32557 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.288589 32482 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.289676 32408 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.479213 32292 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:49.565550 32508 log.cc:924] Time spent T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.917s user 0.007s sys 0.003s
W20260809 07:32:49.384394 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 260362us
I20260809 07:32:49.287150 32534 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.362463 32549 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.301168 32485 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:32:49.417232 32555 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.357s user 0.000s sys 0.001s
I20260809 07:32:49.789526 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:49.143023 32398 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.664095 32505 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:49.332369 32353 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.146751 32397 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:49.297106 32545 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:49.364202 32409 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.671566 32304 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:49.671283 32486 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.650983 32331 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:49.425034 32350 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.059234 32571 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.529157 32389 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.196583 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:49.312652 32540 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.125558 32481 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.509794 32577 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.673173 32546 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.055856 30890 rpcz_store.cc:276] 0809 07:32:47.852565 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.853912 (+ 1347us) service_pool.cc:227] Handling call
0809 07:32:48.960461 (+1106549us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:49.532205 32343 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.705711 30900 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:49.793040 31051 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
W20260809 07:32:49.668839 32401 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.374s user 0.005s sys 0.022s
I20260809 07:32:49.803826 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:49.821664 32511 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.665426 32387 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.796535 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:49.824940 32001 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:49.712340 30899 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s user 0.005s sys 0.019s
I20260809 07:32:49.829427 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:49.792645 31054 rpcz_store.cc:276] 0809 07:32:47.892689 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.892878 (+ 189us) service_pool.cc:227] Handling call
0809 07:32:49.252823 (+1359945us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400344c54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:49.252981 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":538807552}
I20260809 07:32:49.715440 32483 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.830432 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 407) took 1865 ms. Trace:
I20260809 07:32:49.672158 32559 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.745388 32499 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.782135 32490 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.793802 31058 rpcz_store.cc:276] 0809 07:32:48.025290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.132391 (+107101us) service_pool.cc:227] Handling call
0809 07:32:48.186195 (+ 53804us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:48.186434 (+ 239us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:32:48.227872 (+ 41438us) log.cc:842] Serialized 14 byte log entry
0809 07:32:49.364810 (+1136938us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b18001a9080. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b8057b 00007f4094f0388a
0809 07:32:49.653323 (+288513us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:49.653369 (+ 46us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:49.653408 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:49.653473 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:49.653587 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":50764,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":15360,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":4180,"spinlock_wait_cycles":10610816,"thread_start_us":279195,"threads_started":1}
I20260809 07:32:49.849534 32596 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.807049 32408 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.849609 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:49.827324 31003 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.540s user 0.020s sys 0.016s
W20260809 07:32:50.055374 31003 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.770s user 0.021s sys 0.030s
I20260809 07:32:49.788902 32319 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.798638 32475 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:49.696543 32556 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.796979 32536 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:49.197691 32269 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.826798 32476 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:49.893379 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:49.796550 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:49.824790 32303 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.744570 32566 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.812510 32534 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.803720 32491 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.804455 32485 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.827733 32398 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.973022 32350 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.829650 32482 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.054533 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 423) took 2415 ms. Trace:
I20260809 07:32:50.014891 32452 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:49.831068 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 437) took 1108 ms. Trace:
I20260809 07:32:50.055507 31048 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
W20260809 07:32:50.056106 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 259785us
W20260809 07:32:50.057245 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 229235us
I20260809 07:32:49.980080 32409 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.057708 31174 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.057471 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 427) took 1046 ms. Trace:
I20260809 07:32:50.067003 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 413) took 2009 ms. Trace:
I20260809 07:32:50.115491 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:50.116765 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:50.117584 31057 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.118213 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:50.056517 32587 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.367s user 0.000s sys 0.001s
I20260809 07:32:50.056548 30911 rpcz_store.cc:276] 0809 07:32:47.965079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.965185 (+ 106us) service_pool.cc:227] Handling call
0809 07:32:48.958018 (+992833us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:48.958203 (+ 185us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:48.958633 (+ 430us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:48.958659 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:48.959102 (+ 443us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.975912 (+ 16810us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.975958 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.976023 (+ 65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:49.825210 (+849187us) raft_consensus.cc:1671] finished
0809 07:32:49.825230 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:49.830353 (+ 5123us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3307,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"spinlock_wait_cycles":10452224,"thread_start_us":16169,"threads_started":1,"wal-append.queue_time_us":130985}
I20260809 07:32:50.083356 32554 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:49.673502 32459 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:50.067744 32401 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.773s user 0.012s sys 0.032s
W20260809 07:32:50.057912 30899 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.930s user 0.005s sys 0.028s
I20260809 07:32:50.107425 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.128597 31054 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:49.983258 32292 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.068758 32343 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:50.072362 32483 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.975807 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 426) took 2218 ms. Trace:
I20260809 07:32:50.109711 31165 rpcz_store.cc:276] 0809 07:32:47.639492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.645620 (+ 6128us) service_pool.cc:227] Handling call
0809 07:32:47.645913 (+ 293us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.648019 (+ 2106us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.648049 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.648473 (+ 424us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.222739 (+574266us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.222788 (+ 49us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.222857 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:49.476868 (+1254011us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:50.026407 (+549539us) raft_consensus.cc:1671] finished
0809 07:32:50.026433 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.054441 (+ 28008us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1715,"raft.queue_time_us":4250,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2977,"spinlock_wait_cycles":8667392,"thread_start_us":550897,"threads_started":1,"wal-append.queue_time_us":702001}
I20260809 07:32:50.071950 32536 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.999541 32562 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.110072 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.072707 32397 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:50.078714 32579 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.071692 32476 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.078660 32586 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:49.987252 32576 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:50.014394 32461 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:49.922235 32253 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:50.079070 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 398) took 2486 ms. Trace:
W20260809 07:32:50.057071 31051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 580) took 1497 ms (client timeout 1578 ms). Trace:
W20260809 07:32:49.971968 30852 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.66:36075 timed out after 1.578s (SENT) [suppressed 3 similar messages]
I20260809 07:32:49.923374 32331 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.125345 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:49.887122 32547 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.072108 32475 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.080330 32499 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.086687 32577 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.120679 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.182626 32343 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.241632 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.183802 32599 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:49.835150 32557 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.082402 32556 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.185070 32265 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.077172 32387 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.117128 30900 rpcz_store.cc:276] 0809 07:32:49.010634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.679869 (+669235us) service_pool.cc:227] Handling call
0809 07:32:49.705256 (+ 25387us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.057154 (+351898us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.057421 (+ 267us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.250417 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.250982 32387 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.251709 31175 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:50.252112 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.253558 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.189561 31174 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:50.254577 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 533) took 1284 ms. Trace:
I20260809 07:32:50.254951 31174 rpcz_store.cc:276] 0809 07:32:48.969852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.803557 (+833705us) service_pool.cc:227] Handling call
0809 07:32:50.254517 (+450960us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.256011 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.117197 30904 rpcz_store.cc:276] 0809 07:32:48.057715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.215605 (+157890us) service_pool.cc:227] Handling call
0809 07:32:48.243793 (+ 28188us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:48.295149 (+ 51356us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:48.295177 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:48.560886 (+265709us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.944361 (+383475us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.944398 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.944457 (+ 59us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.031603 (+1087146us) raft_consensus.cc:1671] finished
0809 07:32:50.031626 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.066947 (+ 35321us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":379511,"raft.queue_time_us":46985,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":19495,"thread_start_us":271530,"threads_started":1,"wal-append.queue_time_us":357464}
I20260809 07:32:50.256672 31174 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:50.135139 32554 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:32:50.190574 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 73963.9us
I20260809 07:32:50.257122 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 532) took 1289 ms. Trace:
I20260809 07:32:50.115579 32511 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.191341 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:50.182560 32524 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 1.513s user 0.006s sys 0.006s
I20260809 07:32:50.238790 32253 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:49.972030 32304 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:50.262132 32597 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.168s user 0.000s sys 0.001s
I20260809 07:32:50.261397 31120 leader_election.cc:398] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:50.263731 31048 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:50.263952 31120 leader_election.cc:401] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:32:49.890400 32549 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:50.129024 31172 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 534) took 1153 ms (client timeout 1505 ms). Trace:
I20260809 07:32:49.979264 32571 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.227912 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.189003 31046 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:50.192571 32397 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:50.231262 32398 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.255429 32486 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.233577 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.108987 32303 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.127188 32316 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260809 07:32:50.235160 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 263217us
I20260809 07:32:50.289420 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 446) took 1313 ms. Trace:
I20260809 07:32:50.120146 32605 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:50.182346 32578 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:50.233229 31051 rpcz_store.cc:269] 0809 07:32:48.559231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.618008 (+ 58777us) service_pool.cc:227] Handling call
0809 07:32:50.057011 (+1439003us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:50.238222 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 120148us
I20260809 07:32:50.116467 32350 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.118528 32452 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.183190 32537 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.170s
I20260809 07:32:50.126850 32353 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.244385 32408 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.234889 32476 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.115980 31170 rpcz_store.cc:276] 0809 07:32:48.715778 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.721680 (+ 5902us) service_pool.cc:227] Handling call
0809 07:32:49.795086 (+1073406us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b540004d654. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:49.804880 (+ 9794us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11705728}
I20260809 07:32:50.189535 32292 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:50.304709 32501 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 2.048s user 0.005s sys 0.010s
I20260809 07:32:50.289354 32316 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.249058 32599 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.245388 32576 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.257450 31174 rpcz_store.cc:276] 0809 07:32:48.968163 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.255756 (+1287593us) service_pool.cc:227] Handling call
0809 07:32:50.257075 (+ 1319us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.186924 31177 rpcz_store.cc:276] 0809 07:32:47.706716 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.716322 (+ 9606us) service_pool.cc:227] Handling call
0809 07:32:47.716626 (+ 304us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.716944 (+ 318us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.716971 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.717378 (+ 407us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.040363 (+322985us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.040406 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.040483 (+ 77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:49.480944 (+1440461us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:49.851406 (+370462us) raft_consensus.cc:1671] finished
0809 07:32:49.851429 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:49.925496 (+ 74067us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":328,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":731305216,"thread_start_us":322618,"threads_started":1,"wal-append.queue_time_us":576387}
I20260809 07:32:50.071864 32319 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.258764 32554 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:32:50.258901 32602 log.cc:924] Time spent T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.136s user 0.000s sys 0.000s
I20260809 07:32:50.259169 30900 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:50.261144 30911 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:49.789965 30898 rpcz_store.cc:276] 0809 07:32:47.880218 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.565289 (+1685071us) service_pool.cc:227] Handling call
0809 07:32:49.565688 (+ 399us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:49.565893 (+ 205us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:49.565921 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:49.565967 (+ 46us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:49.566051 (+ 84us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:49.566116 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:49.566252 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.079021 32596 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:50.275463 31057 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:50.255873 32265 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:50.231618 32586 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.317483 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.318023 31057 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.290349 31165 rpcz_store.cc:276] 0809 07:32:48.976157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.192203 (+1216046us) service_pool.cc:227] Handling call
0809 07:32:50.289364 (+ 97161us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.292956 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.265761 32491 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.294638 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.276553 32397 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.300685 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.278988 32534 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.280979 32269 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.121368 32409 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:50.267733 31172 rpcz_store.cc:269] 0809 07:32:48.975137 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.047653 (+ 72516us) service_pool.cc:227] Handling call
0809 07:32:50.125298 (+1077645us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.299893 32398 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.247375 32579 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.312556 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.334542 30900 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:50.302469 32387 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260809 07:32:50.302042 32353 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.239589 32562 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.315272 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.340607 31174 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:50.341066 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 538) took 1006 ms. Trace:
I20260809 07:32:50.341379 31174 rpcz_store.cc:276] 0809 07:32:49.334392 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.315043 (+980651us) service_pool.cc:227] Handling call
0809 07:32:50.341016 (+ 25973us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.301755 32578 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.342149 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:50.342661 31174 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.319036 31051 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:50.322847 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.304085 32481 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.326260 30852 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.349058 32511 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.349433 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.349711 32562 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.309418 32316 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.350407 32566 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:50.350890 32566 raft_consensus.cc:686] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:50.329404 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.331987 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:50.308724 32461 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:50.314256 32554 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.296595 32605 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.310746 32459 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:32:50.317133 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:50.345780 32606 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.149s user 0.001s sys 0.000s
I20260809 07:32:50.339139 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.339401 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.320791 32576 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.324867 32547 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.327768 32482 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.346822 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 419) took 2776 ms. Trace:
I20260809 07:32:50.331907 32490 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:32:50.340077 32513 log.cc:924] Time spent T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.776s user 0.007s sys 0.008s
I20260809 07:32:50.349790 32610 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.310374 32485 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.311573 32599 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.352685 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.322475 32265 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.337821 32398 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.355545 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.338984 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.358632 32353 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.359138 32557 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.359586 32303 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.360149 32253 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.361119 32331 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.362129 32319 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.338045 32571 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.325625 32505 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.456503 32577 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:50.494748 30891 consensus_meta.cc:287] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.654s user 0.008s sys 0.002s
I20260809 07:32:50.564805 32609 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.565469 32609 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.513195 32566 consensus_queue.cc:237] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.566048 32609 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.350903 32304 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.491215 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 422) took 1524 ms. Trace:
I20260809 07:32:50.235162 30896 rpcz_store.cc:276] 0809 07:32:47.585958 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.586506 (+ 548us) service_pool.cc:227] Handling call
0809 07:32:47.642548 (+ 56042us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.645752 (+ 3204us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.645786 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.697847 (+ 52061us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b2c00012038. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:32:47.734928 (+ 37081us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.191054 (+456126us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.216456 (+ 25402us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.228028 (+ 11572us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:49.470705 (+1242677us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:50.030403 (+559698us) raft_consensus.cc:1671] finished
0809 07:32:50.054703 (+ 24300us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.069871 (+ 15168us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1356,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":16777856,"thread_start_us":387264,"threads_started":1,"wal-append.queue_time_us":543815}
I20260809 07:32:50.506583 32481 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.534765 31174 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:50.349646 32511 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.568135 31165 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:50.568480 30909 rpcz_store.cc:276] 0809 07:32:48.966851 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.135216 (+168365us) service_pool.cc:227] Handling call
0809 07:32:49.256183 (+120967us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:49.648259 (+392076us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:49.648290 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:49.661091 (+ 12801us) log.cc:842] Serialized 26 byte log entry
0809 07:32:49.832709 (+171618us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:49.832750 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:49.832832 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.403685 (+570853us) raft_consensus.cc:1671] finished
0809 07:32:50.403705 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.491137 (+ 87432us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":78728,"raft.queue_time_us":363272,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":4063,"thread_start_us":116281,"threads_started":1,"wal-append.queue_time_us":385651}
W20260809 07:32:50.567052 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 211584us
I20260809 07:32:50.590893 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 445) took 1617 ms. Trace:
I20260809 07:32:50.589820 31057 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:50.356561 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 415) took 1441 ms. Trace:
I20260809 07:32:50.594439 31174 rpcz_store.cc:276] 0809 07:32:48.973481 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.341945 (+1368464us) service_pool.cc:227] Handling call
0809 07:32:50.590823 (+248878us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:50.478732 32601 log.cc:924] Time spent T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.258s user 0.000s sys 0.001s
I20260809 07:32:50.362838 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.596174 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:50.539822 32533 log.cc:924] Time spent T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 2.033s user 0.016s sys 0.014s
W20260809 07:32:50.404466 32618 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.090s user 0.000s sys 0.001s
I20260809 07:32:50.604930 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.357030 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.404979 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.606379 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:50.629861 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.679989 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:50.671033 31117 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items.
I20260809 07:32:50.685232 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.685933 31054 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:50.685532 31174 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:50.686993 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 543) took 1018 ms. Trace:
I20260809 07:32:50.567730 32490 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:50.568130 31181 rpcz_store.cc:276] 0809 07:32:47.570859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.571492 (+ 633us) service_pool.cc:227] Handling call
0809 07:32:47.571830 (+ 338us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.583070 (+ 11240us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.583099 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.589514 (+ 6415us) log.cc:842] Serialized 26 byte log entry
0809 07:32:47.890700 (+301186us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:47.890755 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:47.890824 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.295940 (+2405116us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:50.342635 (+ 46695us) raft_consensus.cc:1671] finished
0809 07:32:50.342708 (+ 73us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.346748 (+ 4040us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16062,"raft.queue_time_us":12477,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":11024,"spinlock_wait_cycles":10180352,"thread_start_us":300561,"threads_started":1,"wal-append.queue_time_us":562286}
I20260809 07:32:50.388441 30890 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:50.685516 31051 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:50.514860 32578 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.513670 32586 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.689227 31174 rpcz_store.cc:276] 0809 07:32:49.668144 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.604578 (+936434us) service_pool.cc:227] Handling call
0809 07:32:50.686937 (+ 82359us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.686597 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:50.556128 32343 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.693701 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:50.532783 32409 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.439363 32459 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.594326 32511 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.688628 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.385776 32461 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.404644 32617 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:50.700424 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.649952 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 421) took 1680 ms. Trace:
I20260809 07:32:50.702463 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.703904 32626 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:50.705600 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:50.691452 32490 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.568521 32596 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.687263 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:50.686710 32547 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.622118 32482 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.709460 32511 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.718379 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:50.731158 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:50.719173 31051 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.723949 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.724128 32265 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:50.724400 32568 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 1.485s user 0.004s sys 0.005s
I20260809 07:32:50.714658 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 420) took 3143 ms. Trace:
I20260809 07:32:50.786140 31057 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.746625 32398 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.729789 32617 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.789826 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.788862 31164 rpcz_store.cc:276] 0809 07:32:47.571289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.573008 (+ 1719us) service_pool.cc:227] Handling call
0809 07:32:47.584268 (+ 11260us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.637221 (+ 52953us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.639906 (+ 2685us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.640413 (+ 507us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.253547 (+613134us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.257866 (+ 4319us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.278401 (+ 20535us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:48.873785 (+595384us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:50.289267 (+1415482us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b2000063378. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094b8ef11 00007f4094bb0b09 00007f4094b7bc43 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499 00007f4094b84461 00007f4094b8375d 00007f409b235204
0809 07:32:50.699716 (+410449us) raft_consensus.cc:1671] finished
0809 07:32:50.702521 (+ 2805us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.710776 (+ 8255us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":78,"raft.queue_time_us":10122,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7543,"spinlock_wait_cycles":13716608,"thread_start_us":602098,"threads_started":1,"wal-append.queue_time_us":731508}
W20260809 07:32:50.726895 32527 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 2.201s user 0.004s sys 0.022s
I20260809 07:32:50.790632 31057 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.727262 32547 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.727761 32505 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.738515 32475 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.787264 32343 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.746822 32545 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:50.795042 32343 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.795514 32545 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:50.796757 32545 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.797621 32343 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.729012 30896 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.799293 32499 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.598583 32481 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.710052 32459 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.725874 32316 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.807088 31177 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:32:50.734298 32515 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 2.222s user 0.001s sys 0.025s
I20260809 07:32:50.718771 32485 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:50.733052 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:50.809549 32609 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.788408 32596 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:50.810632 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:50.815091 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 82076.8us
W20260809 07:32:50.737222 31046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 591) took 1782 ms (client timeout 1893 ms). Trace:
I20260809 07:32:50.748034 30898 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.740098 30909 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.794662 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 425) took 3101 ms. Trace:
I20260809 07:32:50.801630 32547 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.715101 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 541) took 1287 ms. Trace:
I20260809 07:32:50.809835 32459 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.831243 30852 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.832033 30852 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.832708 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.811125 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.749099 32510 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.740561 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:50.737946 31175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 535) took 1756 ms (client timeout 1597 ms). Trace:
I20260809 07:32:50.816987 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:50.815565 31046 rpcz_store.cc:269] 0809 07:32:48.954296 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.956726 (+ 2430us) service_pool.cc:227] Handling call
0809 07:32:50.737002 (+1780276us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540031bd54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:50.737142 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2503476992}
I20260809 07:32:50.829001 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 590) took 1875 ms. Trace:
I20260809 07:32:50.828068 31182 rpcz_store.cc:276] 0809 07:32:47.693665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.693798 (+ 133us) service_pool.cc:227] Handling call
0809 07:32:47.694120 (+ 322us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.694449 (+ 329us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.694478 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.701555 (+ 7077us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.250563 (+549008us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.250604 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.250672 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.718172 (+2467500us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:50.794378 (+ 76206us) raft_consensus.cc:1671] finished
0809 07:32:50.794403 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.794606 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6424,"raft.queue_time_us":263,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":6479,"spinlock_wait_cycles":5716352,"thread_start_us":511716,"threads_started":1,"wal-append.queue_time_us":806780}
I20260809 07:32:50.838142 30911 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:50.837940 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:50.839118 31055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 568) took 2922 ms (client timeout 2137 ms). Trace:
I20260809 07:32:50.749212 32476 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.746022 32554 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.693998 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 546)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 547)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 545)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 463)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 554)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 555)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 557)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 462)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 470)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 549)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 459)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 458)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 461)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 558)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 556)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 469)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 454)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 560)
I20260809 07:32:50.838944 32562 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.736864 32397 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:50.788097 32482 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:50.845705 31055 rpcz_store.cc:269] 0809 07:32:47.911928 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.912155 (+ 227us) service_pool.cc:227] Handling call
0809 07:32:50.834291 (+2922136us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54004001d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:50.834634 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2344016128}
I20260809 07:32:50.840478 32343 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.828729 31172 rpcz_store.cc:276] 0809 07:32:49.423164 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.342605 (+919441us) service_pool.cc:227] Handling call
0809 07:32:50.710940 (+368335us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.847730 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.837090 32505 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.714696 30891 rpcz_store.cc:276] 0809 07:32:48.963928 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.143724 (+179796us) service_pool.cc:227] Handling call
0809 07:32:49.287052 (+143328us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.578260 (+1291208us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:50.587087 (+ 8827us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:50.592489 (+ 5402us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:50.603303 (+ 10814us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.621866 (+ 18563us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.644323 (+ 22457us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2135,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":23874}
I20260809 07:32:50.817902 32485 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:50.878057 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.887740 31048 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.879202 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.888782 30852 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.880228 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.877260 32547 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.883396 31058 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:50.883507 31051 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:50.883841 31054 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:50.884435 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.888705 31055 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.853113 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:32:50.924132 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 71107.1us
I20260809 07:32:50.924297 31046 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.891947 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.926047 30852 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.926443 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:50.605433 30899 rpcz_store.cc:276] 0809 07:32:48.912560 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.918684 (+ 6124us) service_pool.cc:227] Handling call
0809 07:32:48.947443 (+ 28759us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:48.963071 (+ 15628us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:48.963106 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:49.008624 (+ 45518us) log.cc:842] Serialized 26 byte log entry
0809 07:32:50.193549 (+1184925us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:50.197966 (+ 4417us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.228327 (+ 30361us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.344178 (+115851us) raft_consensus.cc:1671] finished
0809 07:32:50.344194 (+ 16us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.353987 (+ 9793us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5775,"raft.queue_time_us":234,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":2558,"thread_start_us":1011546,"threads_started":1,"wal-append.queue_time_us":971264}
I20260809 07:32:50.929105 32596 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:50.929564 32596 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:50.929461 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.884145 32343 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.943207 31051 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.889976 32485 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:50.889093 32505 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.926687 32482 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.930334 31055 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:50.932572 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 586) took 1105 ms. Trace:
I20260809 07:32:50.934612 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:50.842656 31175 rpcz_store.cc:269] 0809 07:32:48.981546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.701073 (+719527us) service_pool.cc:227] Handling call
0809 07:32:50.737696 (+1036623us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54000c3354. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:50.737880 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":369476864}
I20260809 07:32:50.932904 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:50.897081 32566 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.893230 32536 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:50.930035 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.607s user 0.007s sys 0.008s
W20260809 07:32:50.967494 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.644s user 0.008s sys 0.008s
I20260809 07:32:50.892851 32588 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:50.938828 32475 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.929986 32609 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.930044 32596 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:51.015895 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 416) took 2103 ms. Trace:
I20260809 07:32:50.946802 30852 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:50.979692 31057 raft_consensus.cc:1261] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:50.954568 31046 rpcz_store.cc:276] 0809 07:32:49.827221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.931738 (+1104517us) service_pool.cc:227] Handling call
0809 07:32:50.932093 (+ 355us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.932302 (+ 209us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:50.932326 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:50.932363 (+ 37us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:50.932389 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.932434 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.932523 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.047637 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 600) took 1057 ms. Trace:
I20260809 07:32:51.053380 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.051673 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.062611 31046 rpcz_store.cc:276] 0809 07:32:49.990098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.046578 (+1056480us) service_pool.cc:227] Handling call
0809 07:32:51.047139 (+ 561us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.047310 (+ 171us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.047333 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.047378 (+ 45us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:51.047406 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.047454 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.047579 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.062114 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 594) took 1215 ms. Trace:
I20260809 07:32:50.953598 32316 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.956122 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:51.092871 31055 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.945307 32343 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.948220 32499 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:50.962853 31048 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:50.935415 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 587) took 1105 ms. Trace:
W20260809 07:32:50.967875 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 650555us
W20260809 07:32:50.968508 32628 log.cc:924] Time spent T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.086s user 0.001s sys 0.000s
I20260809 07:32:50.969095 32546 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.964780 32605 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.953954 32505 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:50.970731 30896 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:50.971956 32554 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.973002 32476 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.974093 32397 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.952803 32461 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:50.952030 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 407) took 3477 ms. Trace:
I20260809 07:32:50.975109 32481 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:51.111238 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.112701 32579 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:51.020876 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 93297.8us
I20260809 07:32:51.018356 30906 rpcz_store.cc:276] 0809 07:32:48.912788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.912889 (+ 101us) service_pool.cc:227] Handling call
0809 07:32:50.813944 (+1901055us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:50.814149 (+ 205us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:50.816242 (+ 2093us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:50.816270 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:50.816681 (+ 411us) log.cc:842] Serialized 26 byte log entry
0809 07:32:50.878302 (+ 61621us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:50.878369 (+ 67us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.878432 (+ 63us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.983074 (+104642us) raft_consensus.cc:1671] finished
0809 07:32:50.983094 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.015799 (+ 32705us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":449,"raft.queue_time_us":671,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":14063,"spinlock_wait_cycles":2407680,"thread_start_us":49143,"threads_started":1,"wal-append.queue_time_us":119121}
I20260809 07:32:51.021607 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:50.943681 32510 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:51.031986 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 598) took 1133 ms. Trace:
I20260809 07:32:51.117233 31054 rpcz_store.cc:276] 0809 07:32:49.898559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.030094 (+1131535us) service_pool.cc:227] Handling call
0809 07:32:51.031396 (+ 1302us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.031601 (+ 205us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.031625 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.031664 (+ 39us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:51.031690 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.031741 (+ 51us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.031924 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:50.844857 31045 rpcz_store.cc:276] 0809 07:32:48.954013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:48.956765 (+ 2752us) service_pool.cc:227] Handling call
0809 07:32:48.960343 (+ 3578us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:49.666253 (+705910us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:49.666285 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:49.666354 (+ 69us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:49.666388 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.816649 (+1150261us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b540034b2d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:50.816703 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.828916 (+ 12213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1613517056}
I20260809 07:32:50.956785 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 589) took 1125 ms. Trace:
I20260809 07:32:51.052591 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 582) took 1256 ms. Trace:
I20260809 07:32:51.114171 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.087766 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 545) took 1344 ms. Trace:
I20260809 07:32:51.159968 31182 rpcz_store.cc:276] 0809 07:32:49.743698 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.051272 (+1307574us) service_pool.cc:227] Handling call
0809 07:32:51.087691 (+ 36419us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.088500 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 546) took 1250 ms. Trace:
W20260809 07:32:51.088789 31177 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 449) took 1260 ms (client timeout 1671 ms). Trace:
I20260809 07:32:51.160634 31181 rpcz_store.cc:276] 0809 07:32:49.837707 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.053123 (+1215416us) service_pool.cc:227] Handling call
0809 07:32:51.088438 (+ 35315us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:51.160749 31177 rpcz_store.cc:269] 0809 07:32:49.828798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.314628 (+485830us) service_pool.cc:227] Handling call
0809 07:32:51.088569 (+773941us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b54003895d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:51.088706 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1046935808}
W20260809 07:32:51.049161 32625 log.cc:924] Time spent T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.324s user 0.000s sys 0.004s
I20260809 07:32:50.964316 32483 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.966290 32566 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:50.981676 32617 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.983582 32556 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.112124 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 452) took 1219 ms. Trace:
I20260809 07:32:51.211876 31174 rpcz_store.cc:276] 0809 07:32:49.892837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.110913 (+1218076us) service_pool.cc:227] Handling call
0809 07:32:51.112066 (+ 1153us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.212909 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.213670 31174 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:51.153854 31051 rpcz_store.cc:276] 0809 07:32:49.831083 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.943903 (+1112820us) service_pool.cc:227] Handling call
0809 07:32:50.956283 (+ 12380us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.956491 (+ 208us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:50.956518 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:50.956558 (+ 40us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:50.956586 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.956637 (+ 51us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.956731 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.055825 32562 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:51.158799 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 602) took 1065 ms. Trace:
W20260809 07:32:51.159173 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 212424us
I20260809 07:32:51.161578 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.067096 31057 rpcz_store.cc:276] 0809 07:32:49.846990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.967671 (+1120681us) service_pool.cc:227] Handling call
0809 07:32:50.979232 (+ 11561us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.061758 (+ 82526us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.062047 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.094657 32499 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:51.176452 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:51.172982 32566 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.215133 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.217973 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.153600 32579 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:51.163663 32605 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:51.566051 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 554) took 1307 ms. Trace:
I20260809 07:32:51.569242 31182 rpcz_store.cc:276] 0809 07:32:50.264616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.160517 (+895901us) service_pool.cc:227] Handling call
0809 07:32:51.565989 (+405472us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.353827 30909 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:51.222352 31046 rpcz_store.cc:276] 0809 07:32:50.093254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.091535 (+998281us) service_pool.cc:227] Handling call
0809 07:32:51.158261 (+ 66726us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.158453 (+ 192us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.158476 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.158513 (+ 37us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:51.158540 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.158636 (+ 96us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.158739 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.570174 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 557) took 1263 ms. Trace:
I20260809 07:32:51.566174 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 599) took 1291 ms. Trace:
I20260809 07:32:51.457813 31055 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:51.335352 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 601) took 1056 ms. Trace:
I20260809 07:32:51.110824 31169 rpcz_store.cc:276] 0809 07:32:47.473236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.488260 (+ 15024us) service_pool.cc:227] Handling call
0809 07:32:47.508185 (+ 19925us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:47.567447 (+ 59262us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:47.571298 (+ 3851us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:47.634021 (+ 62723us) log.cc:842] Serialized 26 byte log entry
0809 07:32:48.107931 (+473910us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:48.111571 (+ 3640us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:48.116170 (+ 4599us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:50.595311 (+2479141us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:50.890513 (+295202us) raft_consensus.cc:1671] finished
0809 07:32:50.894457 (+ 3944us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.948028 (+ 53571us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6351,"raft.queue_time_us":30579,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":9332,"spinlock_wait_cycles":5669760,"thread_start_us":417581,"threads_started":1,"wal-append.queue_time_us":683046}
I20260809 07:32:51.157192 31049 rpcz_store.cc:276] 0809 07:32:49.823958 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:49.833213 (+ 9255us) service_pool.cc:227] Handling call
0809 07:32:49.987833 (+154620us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.385939 (+398106us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:50.386879 (+ 940us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:50.403665 (+ 16786us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:50.404604 (+ 939us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.052141 (+647537us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540033a4d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:51.052211 (+ 70us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.052505 (+ 294us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":841276160}
I20260809 07:32:51.573720 30852 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:51.574060 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 606) took 1282 ms. Trace:
I20260809 07:32:51.571372 31048 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:51.571278 31051 rpcz_store.cc:276] 0809 07:32:50.278595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.214664 (+936069us) service_pool.cc:227] Handling call
0809 07:32:51.334856 (+120192us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.335038 (+ 182us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.335062 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.335101 (+ 39us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:51.335125 (+ 24us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.335174 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.335292 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.575752 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 607) took 1283 ms. Trace:
I20260809 07:32:51.576143 31051 rpcz_store.cc:276] 0809 07:32:50.292055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.574835 (+1282780us) service_pool.cc:227] Handling call
0809 07:32:51.575336 (+ 501us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.575491 (+ 155us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.575515 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.575551 (+ 36us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:51.575576 (+ 25us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.575619 (+ 43us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.575703 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.578483 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:51.579231 32397 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:51.249790 32562 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.579706 32397 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:51.580704 32397 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.557648 32505 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.584352 32505 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.250876 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 547) took 1323 ms. Trace:
I20260809 07:32:51.585081 31164 rpcz_store.cc:276] 0809 07:32:49.971914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.113718 (+1141804us) service_pool.cc:227] Handling call
0809 07:32:51.250801 (+137083us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.585829 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 604) took 1294 ms. Trace:
I20260809 07:32:51.586024 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.547519 32549 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:51.569773 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 462) took 1306 ms. Trace:
I20260809 07:32:51.570626 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 601) took 1271 ms. Trace:
I20260809 07:32:51.586410 31057 rpcz_store.cc:276] 0809 07:32:50.291294 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.574226 (+1282932us) service_pool.cc:227] Handling call
0809 07:32:51.585203 (+ 10977us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.585413 (+ 210us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.585438 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.585478 (+ 40us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:51.585504 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.585622 (+ 118us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.585777 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.570703 31177 rpcz_store.cc:276] 0809 07:32:50.306253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.161564 (+855311us) service_pool.cc:227] Handling call
0809 07:32:51.570117 (+408553us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.588181 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.592345 31174 rpcz_store.cc:276] 0809 07:32:50.263293 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.214233 (+950940us) service_pool.cc:227] Handling call
0809 07:32:51.569717 (+355484us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.590550 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 605) took 1299 ms. Trace:
I20260809 07:32:51.571044 31054 rpcz_store.cc:276] 0809 07:32:50.274737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.118131 (+843394us) service_pool.cc:227] Handling call
0809 07:32:51.166473 (+ 48342us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.565837 (+399364us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.565872 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.565916 (+ 44us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:51.565946 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.566002 (+ 56us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.566117 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.210187 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 456) took 1129 ms. Trace:
I20260809 07:32:51.602862 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.566089 31181 raft_consensus.cc:2379] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:51.592492 30909 rpcz_store.cc:276] 0809 07:32:50.298888 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.739577 (+440689us) service_pool.cc:227] Handling call
0809 07:32:50.739864 (+ 287us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.570355 (+830491us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.570571 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.597324 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 610) took 1294 ms. Trace:
I20260809 07:32:51.598498 31046 rpcz_store.cc:276] 0809 07:32:50.291483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.574420 (+1282937us) service_pool.cc:227] Handling call
0809 07:32:51.586875 (+ 12455us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.587033 (+ 158us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.587055 (+ 22us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.590295 (+ 3240us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:51.590358 (+ 63us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.590413 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.590501 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:51.579371 32610 consensus_meta.cc:287] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.983s user 0.007s sys 0.018s
I20260809 07:32:51.607201 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 470) took 1260 ms. Trace:
I20260809 07:32:51.595897 31055 rpcz_store.cc:276] 0809 07:32:50.291741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.573301 (+1281560us) service_pool.cc:227] Handling call
0809 07:32:51.573620 (+ 319us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.573791 (+ 171us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.573813 (+ 22us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.573850 (+ 37us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:51.573877 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.573927 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.574015 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.602286 32331 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:51.643597 32331 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:51.644006 32331 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.645520 32566 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.561957 32343 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:51.646921 31058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 588) took 1816 ms (client timeout 1863 ms). Trace:
I20260809 07:32:51.647668 32343 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.592034 30852 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:51.648442 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 56476.4us
W20260809 07:32:51.649333 32490 consensus_meta.cc:287] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.921s user 0.031s sys 0.062s
I20260809 07:32:51.649895 32609 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:51.649688 32490 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.569463 32499 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.334842 30898 raft_consensus.cc:1261] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:51.169234 32510 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.655098 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.574231 32605 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:51.607663 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.329538 32547 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:51.601289 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.659749 31174 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:51.102368 31059 rpcz_store.cc:276] 0809 07:32:49.830416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.934463 (+1104047us) service_pool.cc:227] Handling call
0809 07:32:50.934855 (+ 392us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.935047 (+ 192us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:50.935080 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:50.935133 (+ 53us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:50.935170 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:50.935238 (+ 68us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:50.935345 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.661104 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.662351 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.662972 31059 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:51.663805 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.664901 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.637424 32610 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:51.666419 31045 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:51.604794 31051 rpcz_store.cc:276] 0809 07:32:50.303220 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.576944 (+1273724us) service_pool.cc:227] Handling call
0809 07:32:51.596893 (+ 19949us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.597047 (+ 154us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:51.597071 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:51.597110 (+ 39us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:51.597136 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.597183 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:51.597273 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.636050 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.418529 32545 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:51.638330 31182 rpcz_store.cc:276] 0809 07:32:50.346404 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.570382 (+1223978us) service_pool.cc:227] Handling call
0809 07:32:51.607146 (+ 36764us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.638674 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.638839 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.636986 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.640777 32556 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 07:32:51.641409 31118 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. [suppressed 1 similar messages]
W20260809 07:32:51.658098 32511 consensus_meta.cc:287] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.868s user 0.004s sys 0.016s
W20260809 07:32:51.608171 32485 consensus_meta.cc:287] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.653s user 0.012s sys 0.006s
I20260809 07:32:51.605049 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 555) took 1337 ms. Trace:
W20260809 07:32:51.647387 31058 rpcz_store.cc:269] 0809 07:32:49.830738 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.080194 (+249456us) service_pool.cc:227] Handling call
0809 07:32:51.646699 (+1566505us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b5400031454. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:51.646859 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1597939584}
I20260809 07:32:51.586879 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 455) took 1524 ms. Trace:
I20260809 07:32:51.586299 31049 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:51.652798 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.589432 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.590763 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.586913 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:51.650251 32609 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:51.656908 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.660282 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 468) took 1333 ms. Trace:
I20260809 07:32:51.687417 31174 rpcz_store.cc:276] 0809 07:32:50.326796 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.598241 (+1271445us) service_pool.cc:227] Handling call
0809 07:32:51.660228 (+ 61987us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:51.591096 32599 consensus_meta.cc:287] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.984s user 0.015s sys 0.006s
W20260809 07:32:51.667403 31165 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 544) took 1977 ms (client timeout 2231 ms). Trace:
I20260809 07:32:51.670300 31118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 560) took 1315 ms. Trace:
I20260809 07:32:51.603307 31175 rpcz_store.cc:276] 0809 07:32:50.080548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.978441 (+897893us) service_pool.cc:227] Handling call
0809 07:32:51.210110 (+231669us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:51.642975 32596 consensus_meta.cc:287] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.627s user 0.014s sys 0.013s
I20260809 07:32:51.670838 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.651079 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.573094 32579 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.671684 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 598) took 1380 ms. Trace:
I20260809 07:32:51.659498 32605 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.681591 30906 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:51.689167 32609 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.740507 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.716944 32343 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:51.684230 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.681648 31181 rpcz_store.cc:276] 0809 07:32:50.267817 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.161349 (+893532us) service_pool.cc:227] Handling call
0809 07:32:51.604994 (+443645us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.687898 31051 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:51.681375 31177 raft_consensus.cc:2396] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:51.714109 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:51.668066 31059 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:51.718021 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.788743 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:51.792693 31174 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:51.793002 30906 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:51.793174 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 562) took 1271 ms. Trace:
I20260809 07:32:51.793615 31174 rpcz_store.cc:276] 0809 07:32:50.522147 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.717623 (+1195476us) service_pool.cc:227] Handling call
0809 07:32:51.793111 (+ 75488us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.670480 31170 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
W20260809 07:32:51.689175 31165 rpcz_store.cc:269] 0809 07:32:49.690500 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.654791 (+1964291us) service_pool.cc:227] Handling call
0809 07:32:51.667341 (+ 12550us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.719237 31118 rpcz_store.cc:276] 0809 07:32:50.355201 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.670256 (+1315055us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 07:32:51.720687 31120 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:51.795310 31177 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 549) took 1715 ms (client timeout 2204 ms). Trace:
I20260809 07:32:51.682967 32485 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.797956 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.670203 32547 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:51.726054 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 591) took 1743 ms. Trace:
I20260809 07:32:51.722555 32596 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.681773 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.779243 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 598) took 1504 ms. Trace:
I20260809 07:32:51.738904 32292 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.689915 30896 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:51.796499 30909 raft_consensus.cc:2379] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:32:51.815594 31182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 459) took 1623 ms (client timeout 2111 ms). Trace:
I20260809 07:32:51.797163 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.808457 31048 rpcz_store.cc:276] 0809 07:32:50.274492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.113509 (+839017us) service_pool.cc:227] Handling call
0809 07:32:51.571216 (+457707us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.574579 (+ 3363us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.757821 (+183242us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54000195d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:51.779185 (+ 21364us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1051818752}
I20260809 07:32:51.804591 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 469)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 566)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 454)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 472)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 570)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 572)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 573)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 574)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 577)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 583)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 579)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 580)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
I20260809 07:32:51.680847 31164 rpcz_store.cc:276] 0809 07:32:50.062947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.585768 (+1522821us) service_pool.cc:227] Handling call
0809 07:32:51.586824 (+ 1056us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.683295 32545 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.738325 32562 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.800429 32499 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:51.805315 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 84743.2us
I20260809 07:32:51.843875 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:51.805826 31174 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
W20260809 07:32:51.846597 31174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 461) took 1614 ms (client timeout 2092 ms). Trace:
W20260809 07:32:51.846952 31174 rpcz_store.cc:269] 0809 07:32:50.232478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.794296 (+1561818us) service_pool.cc:227] Handling call
0809 07:32:51.846536 (+ 52240us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.847877 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.848615 31174 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:51.849102 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 469) took 1518 ms. Trace:
I20260809 07:32:51.849390 31174 rpcz_store.cc:276] 0809 07:32:50.331124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.847625 (+1516501us) service_pool.cc:227] Handling call
0809 07:32:51.849050 (+ 1425us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.850221 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:51.805127 31177 rpcz_store.cc:269] 0809 07:32:50.080044 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.595806 (+1515762us) service_pool.cc:227] Handling call
0809 07:32:51.795243 (+199437us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.715713 32599 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.722206 32481 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:51.809252 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.742050 30891 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:51.674666 32511 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.808190 30911 rpcz_store.cc:276] 0809 07:32:49.982362 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.717962 (+735600us) service_pool.cc:227] Handling call
0809 07:32:50.730552 (+ 12590us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:50.850506 (+119954us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.725788 (+875282us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54003b9f54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:51.725988 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":247012096}
I20260809 07:32:51.810886 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:51.810985 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 161523us
W20260809 07:32:51.814730 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 175684us
I20260809 07:32:51.817651 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:51.819092 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.756646 32350 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:50.701131 31117 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 546)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 547)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 545)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 463)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 554)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 555)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 557)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 462)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 470)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 549)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 459)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 458)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 461)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 558)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 556)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 469)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 454)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 560)
I20260809 07:32:51.819751 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.816581 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 558) took 1502 ms. Trace:
I20260809 07:32:51.877381 31181 rpcz_store.cc:276] 0809 07:32:50.314131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.796890 (+1482759us) service_pool.cc:227] Handling call
0809 07:32:51.816527 (+ 19637us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.757637 32452 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.794580 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.841183 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40582 (request call id 147) took 4164 ms. Trace:
I20260809 07:32:51.809988 32269 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:51.842978 30909 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
W20260809 07:32:51.910108 31165 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 556) took 1612 ms (client timeout 2125 ms). Trace:
W20260809 07:32:51.910868 31165 rpcz_store.cc:269] 0809 07:32:50.297333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.804373 (+1507040us) service_pool.cc:227] Handling call
0809 07:32:51.909997 (+105624us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.779230 32343 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.859119 30745 catalog_manager.cc:5495] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:51.905046 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:51.816066 31182 rpcz_store.cc:269] 0809 07:32:50.191971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.671019 (+1479048us) service_pool.cc:227] Handling call
0809 07:32:51.815540 (+144521us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.738371 30898 rpcz_store.cc:276] 0809 07:32:50.291818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.738066 (+446248us) service_pool.cc:227] Handling call
0809 07:32:50.747861 (+ 9795us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.655363 (+907502us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.670609 (+ 15246us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.916139 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:51.917050 30745 catalog_manager.cc:5495] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:32:51.904203 31055 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:51.875990 30904 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:51.907114 30744 rpcz_store.cc:276] 0809 07:32:47.657757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:47.666110 (+ 8353us) service_pool.cc:227] Handling call
0809 07:32:51.821330 (+4155220us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:32:48.900089 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:32:48.900797 (+ 708us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:32:48.900833 (+ 36us) write_op.cc:435] Acquired schema lock
0809 07:32:48.900851 (+ 18us) tablet.cc:660] Decoding operations
0809 07:32:48.901488 (+ 637us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:32:48.901571 (+ 83us) write_op.cc:641] Partition lock acquired for write op
0809 07:32:48.901596 (+ 25us) tablet.cc:683] Acquiring locks for 4 operations
0809 07:32:48.902050 (+ 454us) tablet.cc:699] Row locks acquired
0809 07:32:48.902067 (+ 17us) write_op.cc:260] PREPARE: finished
0809 07:32:48.902212 (+ 145us) write_op.cc:270] Start()
0809 07:32:48.914375 (+ 12163us) write_op.cc:276] Timestamp: P: 1786260768902166 usec, L: 0
0809 07:32:48.918708 (+ 4333us) op_driver.cc:331] REPLICATION: starting
0809 07:32:48.976836 (+ 58128us) log.cc:842] Serialized 1693 byte log entry
0809 07:32:50.003247 (+1026411us) op_driver.cc:444] REPLICATION: finished
0809 07:32:50.179655 (+176408us) write_op.cc:301] APPLY: starting
0809 07:32:50.179740 (+ 85us) tablet.cc:1365] starting BulkCheckPresence
0809 07:32:50.179897 (+ 157us) tablet.cc:1368] finished BulkCheckPresence
0809 07:32:50.180019 (+ 122us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:32:50.180690 (+ 671us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:32:50.180890 (+ 200us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0809 07:32:50.180935 (+ 45us) write_op.cc:312] APPLY: finished
0809 07:32:50.181207 (+ 272us) log.cc:842] Serialized 40 byte log entry
0809 07:32:50.181369 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:50.181475 (+ 106us) write_op.cc:454] Released schema lock
0809 07:32:50.181518 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0809 07:32:50.181799 (+ 281us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:50.181824 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":112014,"apply.run_cpu_time_us":2472,"apply.run_wall_time_us":2473,"num_ops":4,"prepare.queue_time_us":450943,"prepare.run_cpu_time_us":31183,"prepare.run_wall_time_us":295138,"raft.queue_time_us":337243,"raft.run_cpu_time_us":21625,"raft.run_wall_time_us":200330,"replication_time_us":1063121,"thread_start_us":764327,"threads_started":4,"wal-append.queue_time_us":441905,"wal-append.run_cpu_time_us":12899,"wal-append.run_wall_time_us":1776868}]]}
I20260809 07:32:51.851199 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 561) took 1453 ms. Trace:
I20260809 07:32:51.920339 31174 rpcz_store.cc:276] 0809 07:32:50.403999 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.850002 (+1446003us) service_pool.cc:227] Handling call
0809 07:32:51.851148 (+ 1146us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.878345 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.914228 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:51.921880 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 564) took 1278 ms. Trace:
I20260809 07:32:51.922194 31181 rpcz_store.cc:276] 0809 07:32:50.643212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.878063 (+1234851us) service_pool.cc:227] Handling call
0809 07:32:51.921825 (+ 43762us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:51.926259 30745 catalog_manager.cc:5495] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:32:51.930727 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.930748 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.860416 32490 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.933508 32599 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.934937 32499 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.935460 32461 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:51.935850 32461 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:51.936200 32461 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:51.938100 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:51.938582 31175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 458) took 1825 ms (client timeout 2206 ms). Trace:
I20260809 07:32:51.938920 32610 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:51.939514 32511 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.877811 32510 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.939785 32292 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:51.940127 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:51.940188 32635 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:51.940732 32635 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:51.941085 32635 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:51.941684 32579 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.943199 32545 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:51.943755 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 605) took 1652 ms. Trace:
I20260809 07:32:51.944123 31045 rpcz_store.cc:276] 0809 07:32:50.291233 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.378047 (+1086814us) service_pool.cc:227] Handling call
0809 07:32:51.654097 (+276050us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:51.669509 (+ 15412us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:51.913942 (+244433us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400031450. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:51.943696 (+ 29754us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":543089920}
I20260809 07:32:51.879048 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.950968 32635 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.944000 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:51.951354 32562 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.905727 32481 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.951896 31059 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:51.952168 32545 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:51.952595 32545 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:51.952920 32545 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:51.953369 32481 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:51.938993 31175 rpcz_store.cc:269] 0809 07:32:50.113131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.738562 (+1625431us) service_pool.cc:227] Handling call
0809 07:32:51.938522 (+199960us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:51.954938 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 44732.6us
I20260809 07:32:51.955726 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:51.815384 32547 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:51.922591 30900 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:51.843071 32566 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:51.957441 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.962307 31046 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:51.922967 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:51.967311 30906 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:51.967376 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:51.969125 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:51.908867 32397 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:51.971035 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.973842 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 567) took 1008 ms. Trace:
I20260809 07:32:51.974396 31174 rpcz_store.cc:276] 0809 07:32:50.965570 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.921112 (+955542us) service_pool.cc:227] Handling call
0809 07:32:51.973739 (+ 52627us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:51.917255 31170 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 531) took 2949 ms (client timeout 2086 ms). Trace:
I20260809 07:32:51.982144 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.987280 31055 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:51.956821 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.964327 30890 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:51.993294 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:51.993407 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:51.994341 31048 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:51.996675 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:51.997444 31174 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:51.973533 32546 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:51.999728 32546 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:51.968060 32331 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.000960 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.002034 32596 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.002034 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.003731 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.004321 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.004652 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.005623 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.006568 31165 raft_consensus.cc:2379] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.007021 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 454) took 2029 ms. Trace:
I20260809 07:32:52.007316 31165 rpcz_store.cc:276] 0809 07:32:49.978102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.911580 (+1933478us) service_pool.cc:227] Handling call
0809 07:32:52.005458 (+ 93878us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:52.006926 (+ 1468us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17299840}
I20260809 07:32:51.973737 32397 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:52.002449 32331 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.008268 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:51.968209 32637 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:51.973696 32549 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:51.989192 31169 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:51.977492 32566 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260809 07:32:52.011373 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.015475 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.030146 31181 raft_consensus.cc:2396] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260809 07:32:52.016865 30900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 603) took 1708 ms (client timeout 1984 ms). Trace:
W20260809 07:32:52.030989 30900 rpcz_store.cc:269] 0809 07:32:50.308023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.587601 (+279578us) service_pool.cc:227] Handling call
0809 07:32:52.016694 (+1429093us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001f0c54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:52.016801 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2848543104}
I20260809 07:32:52.029178 31117 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 573)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 574)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 577)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 480)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 583)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 482)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 589)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 588)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 481)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 579)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 486)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 484)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 475)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 580)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 483)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 477)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 585)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 485)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 476)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 590)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 479)
I20260809 07:32:52.005632 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.035035 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.039216 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 573)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 578)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 574)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 577)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 480)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 581)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 582)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 583)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 482)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 589)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 588)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 591)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 481)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 579)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 486)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 484)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 475)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 580)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 483)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 477)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 585)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 485)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 476)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 590)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
I20260809 07:32:52.023217 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.060245 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.022864 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.063609 31165 raft_consensus.cc:2362] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3. Re-sending same reply.
I20260809 07:32:52.014637 32397 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.027019 31172 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:32:52.075498 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 170293us
I20260809 07:32:52.032958 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.083143 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.077533 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.082839 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.087098 30900 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:32:52.088019 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 128459us
W20260809 07:32:52.023514 31169 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 565) took 1336 ms (client timeout 1578 ms). Trace:
I20260809 07:32:52.088337 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.092360 30909 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:52.024631 32637 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:52.026907 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.093845 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.070061 32639 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:52.096573 31174 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
W20260809 07:32:52.097023 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 124783us
I20260809 07:32:52.090688 31181 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.090756 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.091442 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.098826 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.100014 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.100998 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.025576 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.102886 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.103576 30909 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:52.104452 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.105901 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.102977 31045 raft_consensus.cc:2362] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3. Re-sending same reply.
W20260809 07:32:51.985968 31170 rpcz_store.cc:269] 0809 07:32:48.965413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.305240 (+1339827us) service_pool.cc:227] Handling call
0809 07:32:51.907553 (+1602313us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003f0f54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:51.915230 (+ 7677us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3038142336}
I20260809 07:32:52.101994 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.110930 31165 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:52.076159 32562 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.112658 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.094028 31006 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.116915 31165 raft_consensus.cc:2396] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:52.118840 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.120524 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.115036 31175 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:52.109098 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.123766 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.122653 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.124145 31055 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
W20260809 07:32:52.110306 31172 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 463) took 1837 ms (client timeout 1694 ms). Trace:
I20260809 07:32:52.121507 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.126904 30852 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:52.093119 31169 rpcz_store.cc:269] 0809 07:32:50.687031 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.586685 (+899654us) service_pool.cc:227] Handling call
0809 07:32:52.023446 (+436761us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:52.128250 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.135102 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.116717 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.131642 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.141872 31174 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:52.136358 31006 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.118968 32485 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.130913 30909 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:52.121699 32343 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.119513 32547 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.116477 32562 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.140707 32566 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:52.145428 32637 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:52.146576 32641 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260809 07:32:52.154503 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.123482 32350 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:52.134881 32635 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.176388 31059 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:52.177037 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.127368 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.177824 30891 raft_consensus.cc:2379] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:52.177938 31170 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.178766 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.179945 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.137135 32617 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:52.171355 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.174038 32566 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:52.162962 30898 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:52.174530 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.175280 32350 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:52.178721 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.109921 32639 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.180974 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.175006 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:52.127568 31172 rpcz_store.cc:269] 0809 07:32:50.271151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.819643 (+1548492us) service_pool.cc:227] Handling call
0809 07:32:52.096959 (+277316us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:52.191537 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.146178 32461 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.156965 32610 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260809 07:32:52.251381 30900 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:52.230409 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.221704 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.274866 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.275779 30900 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.228973 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.276521 30904 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.226253 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.162983 32596 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260809 07:32:52.182647 32617 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:52.183961 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.199666 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.278224 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.322974 30911 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.277477 30909 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:52.323726 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:52.257875 32605 consensus_meta.cc:287] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.514s user 0.008s sys 0.001s
I20260809 07:32:52.325227 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.325883 30900 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:52.331269 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:52.335489 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 199153us
I20260809 07:32:52.336557 31006 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.262465 32486 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:52.193514 32657 consensus_queue.cc:1046] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.145s
I20260809 07:32:52.338007 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.338092 32486 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.259425 32350 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.339358 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.339900 30900 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:52.340848 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.256130 32269 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.341890 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.264670 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.221612 32653 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.265516 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.341502 32605 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.337255 32669 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:52.235761 32397 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.199383 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.337324 31006 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.356555 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.422058 30900 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:52.227603 32638 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:32:52.192250 31164 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 559) took 1853 ms (client timeout 2110 ms). Trace:
W20260809 07:32:52.426924 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 89658.8us
I20260809 07:32:52.355284 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:52.427500 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 96315us
I20260809 07:32:52.298784 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.430611 30906 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:52.419387 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.431533 30909 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.234892 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.425369 31177 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:52.432719 30904 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:52.434044 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.435400 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.229815 32639 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.435964 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.436657 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.438163 30911 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.233235 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.231612 32566 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.222090 32547 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:52.251876 30896 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.328197 32617 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.444763 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:52.347805 32562 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.437849 31165 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:52.453651 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:52.456635 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.459510 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:52.459647 32670 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:52.460188 32670 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.460827 30911 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.461328 32546 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:52.364187 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.356635 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:52.354843 32609 consensus_meta.cc:287] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.594s user 0.003s sys 0.007s
I20260809 07:32:52.444789 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:52.453249 31177 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 566) took 1504 ms (client timeout 1809 ms). Trace:
I20260809 07:32:52.349050 32461 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.344516 32486 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.454020 30900 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.438341 31181 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.348248 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.446564 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.453328 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.484138 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:52.476848 30853 connection.cc:232] RPC call timeout handler was delayed by -1.2945884398650378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260809 07:32:52.437003 32269 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:52.458048 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.346673 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.421806 32669 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:52.446249 32547 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:52.472019 31046 raft_consensus.cc:1226] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.477574 31055 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.463843 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.445859 32662 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:52.446082 32653 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.437386 30899 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.434018 32638 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:52.494364 30890 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:52.452855 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.484786 31051 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:52.483778 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:52.484918 30853 connection.cc:232] RPC call timeout handler was delayed by -1.2463681527879089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260809 07:32:52.465193 32485 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:52.462065 32658 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:52.483855 31059 raft_consensus.cc:2379] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:52.446589 31170 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:52.486992 30898 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.345891 32545 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:52.477774 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.490891 31117 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 476)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 590)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 613)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 611)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 604)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 606)
I20260809 07:32:52.494159 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:52.479310 31177 rpcz_store.cc:269] 0809 07:32:50.948885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.851803 (+902918us) service_pool.cc:227] Handling call
0809 07:32:52.453077 (+601274us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54003fded4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:52.453188 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":970611456}
I20260809 07:32:52.485445 32635 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.453528 30891 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
W20260809 07:32:52.434509 31164 rpcz_store.cc:269] 0809 07:32:50.338681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.843315 (+1504634us) service_pool.cc:227] Handling call
0809 07:32:52.115070 (+271755us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99bce 000000000034aea9 00007f4099dc5a30 00007f4099e8b7b4 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:52.188522 (+ 73452us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":92320000}
W20260809 07:32:52.501062 30853 connection.cc:232] RPC call timeout handler was delayed by -1.2292681527324021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260809 07:32:52.501648 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.489296 31172 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.490087 32547 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.483172 32609 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.489863 32670 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.498682 30909 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.567723 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:52.508569 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 44816.5us
I20260809 07:32:52.621944 30909 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:52.622596 31005 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.623797 30909 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.624274 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:52.622090 31117 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 476)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 590)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 613)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 611)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 492)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 604)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 606)
I20260809 07:32:52.625501 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.624943 31181 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
W20260809 07:32:52.620648 30853 leader_election.cc:334] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Timed out: RequestConsensusVote RPC to 127.28.87.67:44107 timed out after 1.505s (SENT)
I20260809 07:32:52.626608 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.535310 31174 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:52.626178 31177 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:52.628760 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.627655 30853 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.631685 30909 raft_consensus.cc:1226] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.626335 31117 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 476)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 590)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 586)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 595)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 613)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 611)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 596)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 492)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 604)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 606)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 493)
I20260809 07:32:52.500407 31169 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.489549 32669 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.628477 31005 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.640235 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.679358 31181 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
W20260809 07:32:52.679734 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 51331.4us
I20260809 07:32:52.682058 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.499291 32657 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.535805 32653 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.657950 31182 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.686332 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.617213 30900 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:52.504683 32481 raft_consensus.cc:3032] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:52.736301 30909 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:52.496541 32662 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:52.737668 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.633183 32476 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:52.565167 32668 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.700011 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.654850 30906 raft_consensus.cc:1226] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.741596 31174 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:52.648768 32397 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.502095 32638 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:52.536343 32605 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.631265 32485 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.743103 32476 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:52.743180 32662 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.667399 32350 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.690443 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.747586 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.746536 32644 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:52.763639 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.750401 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.750948 32545 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:52.751498 31182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 472) took 2158 ms (client timeout 2301 ms). Trace:
W20260809 07:32:52.769475 31182 rpcz_store.cc:269] 0809 07:32:50.592817 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:51.915635 (+1322818us) service_pool.cc:227] Handling call
0809 07:32:52.004185 (+ 88550us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:52.751290 (+747105us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b540004db54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:52.751437 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1590323968}
I20260809 07:32:52.631007 32486 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 07:32:52.746155 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46358.6us
I20260809 07:32:52.774895 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.652580 32661 raft_consensus.cc:3032] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:52.682760 32556 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:52.705899 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 576) took 1117 ms. Trace:
I20260809 07:32:52.632546 32635 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.736057 32481 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:52.764510 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.765429 31181 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.760843 32476 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.761157 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.630647 32549 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.710911 31172 raft_consensus.cc:2362] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3. Re-sending same reply.
W20260809 07:32:52.781560 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 335062us
I20260809 07:32:52.783066 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.766536 32485 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.786830 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.787400 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.789510 31005 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.789834 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:52.790656 32609 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:52.782697 32644 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.792078 31174 raft_consensus.cc:2379] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.817884 32481 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.829152 32481 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.820318 30896 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:52.819269 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.822243 32486 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.820106 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.831902 32635 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.832223 31181 raft_consensus.cc:2354] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:52.832469 32635 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.832984 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.782857 31175 rpcz_store.cc:276] 0809 07:32:51.588877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.436269 (+847392us) service_pool.cc:227] Handling call
0809 07:32:52.686081 (+249812us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:52.833719 31181 raft_consensus.cc:2379] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.780753 32661 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:52.821483 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.837862 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:52.838510 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:52.835268 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.826890 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.818755 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.820469 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.840649 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.842127 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.841233 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 584) took 1200 ms. Trace:
I20260809 07:32:52.843954 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.844635 31177 raft_consensus.cc:2379] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.843812 31174 rpcz_store.cc:276] 0809 07:32:51.641228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.823658 (+1182430us) service_pool.cc:227] Handling call
0809 07:32:52.841172 (+ 17514us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:52.841739 31182 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:52.847911 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.850648 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.851409 31182 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:52.852173 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.853183 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.854431 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.855062 31177 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:52.855381 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.856603 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.857928 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.863749 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.865689 31177 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
W20260809 07:32:52.866537 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 217162us
I20260809 07:32:52.888851 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.888856 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.832938 32635 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.843012 32599 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:52.846460 32486 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:52.892482 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.817893 32549 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.846836 32269 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:52.894141 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:52.631052 32658 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.891184 32461 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:52.898478 32397 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260809 07:32:52.890887 30853 connection.cc:232] RPC call timeout handler was delayed by -1.05016815289855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260809 07:32:52.895243 31006 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.859138 32485 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.902405 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:52.890244 32682 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:52.906462 32610 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:52.907589 31182 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.893014 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:52.910413 30853 connection.cc:232] RPC call timeout handler was delayed by -1.0384681527502835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260809 07:32:52.897305 32556 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:52.859078 32653 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.908761 32486 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:52.928966 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.930025 31174 raft_consensus.cc:2396] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:52.913458 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:52.899689 32670 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260809 07:32:52.894399 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.914449 31181 raft_consensus.cc:2396] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:52.932308 31177 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:52.910547 32397 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.932861 32397 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:52.933570 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.934684 31177 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.935732 31177 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:52.936225 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 530) took 3971 ms. Trace:
I20260809 07:32:52.936566 31177 rpcz_store.cc:276] 0809 07:32:48.965219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.935326 (+3970107us) service_pool.cc:227] Handling call
0809 07:32:52.935582 (+ 256us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:52.936026 (+ 444us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:52.936179 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:52.915158 32599 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:52.915453 32654 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:52.910471 32461 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.916420 32481 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:52.891621 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:52.901163 32269 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.909969 31006 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:53.032269 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 122374us
I20260809 07:32:53.032466 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 537) took 4011 ms. Trace:
I20260809 07:32:53.033263 31006 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:53.033398 31174 rpcz_store.cc:276] 0809 07:32:49.016154 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.939056 (+3922902us) service_pool.cc:227] Handling call
0809 07:32:53.027031 (+ 87975us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.027194 (+ 163us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.027218 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.027268 (+ 50us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:53.027297 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.027345 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.027523 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.034680 31006 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.899618 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.891635 32684 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 07:32:52.923781 30853 connection.cc:232] RPC call timeout handler was delayed by -1.0375681528821588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
I20260809 07:32:53.036399 32599 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:52.892226 32545 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260809 07:32:53.014708 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:52.924301 32610 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.974062 32668 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 07:32:53.015733 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 177264us
I20260809 07:32:52.980904 32635 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.001827 31057 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:52.997772 32486 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.926028 31169 raft_consensus.cc:2362] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3. Re-sending same reply.
I20260809 07:32:52.925515 32556 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:52.910734 32661 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.982584 32549 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:52.910894 30899 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.971874 30891 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:52.942721 32682 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:53.028513 32481 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:52.942683 32476 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:52.980451 32397 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.010964 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:52.991384 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 539) took 3613 ms. Trace:
I20260809 07:32:53.069166 31182 rpcz_store.cc:276] 0809 07:32:49.378175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.990605 (+3612430us) service_pool.cc:227] Handling call
0809 07:32:52.990909 (+ 304us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:52.991085 (+ 176us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:52.991110 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:52.991148 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:52.991176 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:52.991223 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:52.991331 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.027009 32461 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.036398 32269 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.074615 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 548) took 3019 ms. Trace:
I20260809 07:32:53.059671 32556 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.042600 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 450) took 3213 ms. Trace:
I20260809 07:32:53.048794 32545 raft_consensus.cc:3032] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.052482 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:53.053671 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:53.055375 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 540) took 3670 ms. Trace:
I20260809 07:32:53.079280 31174 rpcz_store.cc:276] 0809 07:32:49.385338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.035263 (+3649925us) service_pool.cc:227] Handling call
0809 07:32:53.054893 (+ 19630us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.055075 (+ 182us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.055099 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.055137 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:53.055167 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.055215 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.055309 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.061519 31006 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:52.918174 32547 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.038056 32684 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:53.063862 32682 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.056124 32703 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.101855 32549 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:53.046941 32599 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.116439 32703 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:53.116863 32703 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.071292 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40564 (request call id 147) took 2362 ms. Trace:
I20260809 07:32:53.056363 32485 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.122457 32485 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.122841 32485 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.123922 32461 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.124960 31174 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.078009 32545 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.125813 32545 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.126935 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 457) took 3039 ms. Trace:
I20260809 07:32:53.127382 31165 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:53.127849 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 453) took 3225 ms. Trace:
I20260809 07:32:53.126947 32545 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.128408 32682 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.129251 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 485) took 1123 ms. Trace:
I20260809 07:32:53.129578 31165 rpcz_store.cc:276] 0809 07:32:52.005986 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.486037 (+480051us) service_pool.cc:227] Handling call
0809 07:32:53.129084 (+643047us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54003f1954. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:53.129200 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1154991360}
I20260809 07:32:53.130735 32546 consensus_queue.cc:1046] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.131788 32546 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.132458 32653 consensus_queue.cc:1046] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.133019 32546 raft_consensus.cc:3032] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.127231 32658 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.133397 32546 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.133533 32653 consensus_queue.cc:1046] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.133945 32635 raft_consensus.cc:3032] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.075011 31182 rpcz_store.cc:276] 0809 07:32:50.055632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.074058 (+3018426us) service_pool.cc:227] Handling call
0809 07:32:53.074270 (+ 212us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.074424 (+ 154us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.074439 (+ 15us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.074466 (+ 27us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:53.074481 (+ 15us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.074512 (+ 31us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.074575 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:53.027666 32617 consensus_meta.cc:287] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.501s user 0.004s sys 0.001s
I20260809 07:32:53.134406 32635 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.122889 32476 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.134723 32617 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.135473 32476 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:53.135293 32635 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.135833 32476 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:53.136198 32476 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.136468 32635 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.134862 32657 raft_consensus.cc:3032] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.137871 32657 raft_consensus.cc:729] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.138666 32657 consensus_queue.cc:260] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.134075 31169 rpcz_store.cc:276] 0809 07:32:49.901980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.063809 (+3161829us) service_pool.cc:227] Handling call
0809 07:32:53.071975 (+ 8166us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.127510 (+ 55535us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.127548 (+ 38us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.127591 (+ 43us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:53.127621 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.127684 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.127797 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":28060,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104}
I20260809 07:32:53.127311 31174 rpcz_store.cc:276] 0809 07:32:50.087580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.080676 (+2993096us) service_pool.cc:227] Handling call
0809 07:32:53.080987 (+ 311us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.125357 (+ 44370us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.126675 (+ 1318us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b54000c4254. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:53.126868 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":100713728}
I20260809 07:32:53.140305 31049 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.140880 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 551) took 3013 ms. Trace:
I20260809 07:32:53.141263 31169 rpcz_store.cc:276] 0809 07:32:50.127935 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.140084 (+3012149us) service_pool.cc:227] Handling call
0809 07:32:53.140401 (+ 317us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.140574 (+ 173us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.140600 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.140640 (+ 40us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:53.140672 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.140722 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.140827 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.143301 32549 raft_consensus.cc:686] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.143636 32461 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.144331 32549 consensus_queue.cc:237] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.060441 30911 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.077929 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 592) took 1013 ms. Trace:
I20260809 07:32:53.146567 31164 rpcz_store.cc:276] 0809 07:32:52.064476 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.010672 (+946196us) service_pool.cc:227] Handling call
0809 07:32:53.077859 (+ 67187us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.084113 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:53.087459 32397 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.137449 31182 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.139838 32657 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.148312 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 550) took 3035 ms. Trace:
I20260809 07:32:53.148675 31182 rpcz_store.cc:276] 0809 07:32:50.112616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.137022 (+3024406us) service_pool.cc:227] Handling call
0809 07:32:53.137300 (+ 278us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.148108 (+ 10808us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.148264 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.088421 32638 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.062189 32566 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260809 07:32:53.035533 32669 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.090997 32662 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.068367 32481 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.151140 31059 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.151966 32662 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.091390 32610 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.107165 32668 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:53.152984 32668 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.153744 32566 consensus_queue.cc:1046] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:53.154718 32566 raft_consensus.cc:3032] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.154892 32668 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:53.107342 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 53752us
I20260809 07:32:53.156636 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:53.157601 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:32:53.107967 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46760.1us
I20260809 07:32:53.136668 32617 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.115634 32547 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.106592 32661 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.158896 32617 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.159238 32617 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:53.159612 32617 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.131889 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:53.160836 32610 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:53.161301 32610 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.162096 32610 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.162636 32547 consensus_queue.cc:1046] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.163982 32547 raft_consensus.cc:3032] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.164309 32547 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.164732 30909 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.165069 30911 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.165141 32547 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.165975 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.168150 32668 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.140893 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 552) took 2943 ms. Trace:
I20260809 07:32:53.168936 31174 rpcz_store.cc:276] 0809 07:32:50.205189 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.140163 (+2934974us) service_pool.cc:227] Handling call
0809 07:32:53.140455 (+ 292us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.140608 (+ 153us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.140632 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.140668 (+ 36us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:53.140696 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.140753 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.140844 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.169754 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.169899 31006 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:53.099488 32684 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.152046 32638 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.171666 32638 consensus_queue.cc:1046] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.173400 32668 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:53.173898 32668 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.173974 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.176465 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.176846 32481 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.177263 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.177629 30890 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.178532 31172 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.179095 30896 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.179915 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 451) took 3344 ms. Trace:
I20260809 07:32:53.180330 31172 rpcz_store.cc:276] 0809 07:32:49.835662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.049109 (+3213447us) service_pool.cc:227] Handling call
0809 07:32:53.049420 (+ 311us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.179027 (+129607us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.179666 (+ 639us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540004d3d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:53.179851 (+ 185us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":296442752}
I20260809 07:32:53.027907 32654 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:53.182173 32654 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.109731 32269 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.186270 32638 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.077099 31181 rpcz_store.cc:276] 0809 07:32:49.829262 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.041778 (+3212516us) service_pool.cc:227] Handling call
0809 07:32:53.042129 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.042289 (+ 160us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.042344 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.042382 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:53.042409 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.042457 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.042552 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:53.186810 32638 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:53.178753 30899 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.188045 32638 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.180511 32547 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.192018 32703 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.193261 31057 raft_consensus.cc:1226] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.155093 32566 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.195633 31046 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.196071 32654 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.198474 32669 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.072402 32644 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.239092 30906 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.238022 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.233624 31046 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.118507 30745 rpcz_store.cc:276] 0809 07:32:50.719974 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:50.723586 (+ 3612us) service_pool.cc:227] Handling call
0809 07:32:53.071178 (+2347592us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:32:52.136700 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:32:52.151024 (+ 14324us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:32:52.151062 (+ 38us) write_op.cc:435] Acquired schema lock
0809 07:32:52.151075 (+ 13us) tablet.cc:660] Decoding operations
0809 07:32:52.151614 (+ 539us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:32:52.151663 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0809 07:32:52.151692 (+ 29us) tablet.cc:683] Acquiring locks for 3 operations
0809 07:32:52.152008 (+ 316us) tablet.cc:699] Row locks acquired
0809 07:32:52.152026 (+ 18us) write_op.cc:260] PREPARE: finished
0809 07:32:52.152172 (+ 146us) write_op.cc:270] Start()
0809 07:32:52.152269 (+ 97us) write_op.cc:276] Timestamp: P: 1786260772152129 usec, L: 0
0809 07:32:52.152288 (+ 19us) op_driver.cc:331] REPLICATION: starting
0809 07:32:52.152892 (+ 604us) log.cc:842] Serialized 1307 byte log entry
0809 07:32:52.506251 (+353359us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b18000cb140. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:32:52.506503 (+ 252us) op_driver.cc:444] REPLICATION: finished
0809 07:32:52.535940 (+ 29437us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b18000cb140. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fce155 00007f4094e8ea30 00007f4094e8844d 00007f4094e8778b
0809 07:32:52.681884 (+145944us) write_op.cc:301] APPLY: starting
0809 07:32:52.681960 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0809 07:32:52.682086 (+ 126us) tablet.cc:1368] finished BulkCheckPresence
0809 07:32:52.682105 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:32:52.740123 (+ 58018us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:32:52.764075 (+ 23952us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0809 07:32:52.774480 (+ 10405us) write_op.cc:312] APPLY: finished
0809 07:32:52.842575 (+ 68095us) log.cc:842] Serialized 34 byte log entry
0809 07:32:52.863131 (+ 20556us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:52.889445 (+ 26314us) write_op.cc:454] Released schema lock
0809 07:32:52.890888 (+ 1443us) write_op.cc:341] FINISH: Updating metrics
0809 07:32:52.898921 (+ 8033us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:52.898950 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1094710,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":175110,"apply.run_cpu_time_us":3127,"apply.run_wall_time_us":217340,"num_ops":3,"prepare.queue_time_us":183234,"prepare.run_cpu_time_us":37241,"prepare.run_wall_time_us":196447,"raft.queue_time_us":53720,"raft.run_cpu_time_us":17485,"raft.run_wall_time_us":204576,"replication_time_us":354127,"spinlock_wait_cycles":314162944,"thread_start_us":531478,"threads_started":4,"wal-append.queue_time_us":126943}]]}
I20260809 07:32:53.212638 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 553) took 2970 ms. Trace:
I20260809 07:32:53.234428 30890 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.250921 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.250638 31165 rpcz_store.cc:276] 0809 07:32:50.242392 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.142263 (+2899871us) service_pool.cc:227] Handling call
0809 07:32:53.211909 (+ 69646us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:53.212163 (+ 254us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.212338 (+ 175us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:53.212362 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.212403 (+ 41us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:53.212429 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.212476 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.212581 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8682752}
I20260809 07:32:53.237576 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.237004 32562 consensus_queue.cc:1046] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260809 07:32:53.238548 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.237937 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.237943 32481 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.255391 32350 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:32:53.203383 32269 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.213852 31181 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:53.256600 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.256691 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.257171 32350 raft_consensus.cc:3032] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.257618 32350 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.237183 30899 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.202250 32703 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.215178 31164 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.248307 32546 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.202790 32511 consensus_queue.cc:1046] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260809 07:32:53.239148 32556 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.203639 32638 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.266268 32511 raft_consensus.cc:3032] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.202459 32343 consensus_queue.cc:1046] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260809 07:32:53.252058 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.210820 31172 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:53.270087 31174 raft_consensus.cc:1226] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:32:53.271090 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 467) took 2944 ms. Trace:
I20260809 07:32:53.271386 31174 rpcz_store.cc:276] 0809 07:32:50.326474 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.180924 (+2854450us) service_pool.cc:227] Handling call
0809 07:32:53.264528 (+ 83604us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:53.264709 (+ 181us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:53.270833 (+ 6124us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.271035 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":141022464}
I20260809 07:32:53.261221 32476 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.258736 32350 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.253690 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.259075 32703 raft_consensus.cc:3032] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.225765 32485 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.206518 32654 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.210290 31169 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.260615 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 465) took 2962 ms. Trace:
I20260809 07:32:53.263995 32609 raft_consensus.cc:3032] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.231999 32684 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.283159 32609 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.267539 32546 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.284523 31170 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:32:53.285229 32684 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.283363 32556 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.286237 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 447) took 3458 ms. Trace:
I20260809 07:32:53.286863 31170 rpcz_store.cc:276] 0809 07:32:49.827531 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.038278 (+3210747us) service_pool.cc:227] Handling call
0809 07:32:53.050296 (+ 12018us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:53.285157 (+234861us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.285945 (+ 788us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002698d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:53.286173 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":537835392}
I20260809 07:32:53.289242 32350 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.284358 32609 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.292456 32485 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.293160 32609 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.294814 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 460) took 3091 ms. Trace:
I20260809 07:32:53.295713 31046 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.273540 32511 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.228928 32599 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.307350 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:53.315265 30853 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.316417 30853 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.280429 32703 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:53.329737 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 536) took 4319 ms. Trace:
I20260809 07:32:53.330166 31177 rpcz_store.cc:276] 0809 07:32:49.011590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.937418 (+3925828us) service_pool.cc:227] Handling call
0809 07:32:52.937697 (+ 279us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:52.937858 (+ 161us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:32:53.041711 (+103853us) log.cc:842] Serialized 14 byte log entry
0809 07:32:53.329398 (+287687us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:32:53.329450 (+ 52us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:53.329502 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.329565 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:53.329681 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":50270,"prepare.queue_time_us":308084,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":340552,"threads_started":2}
I20260809 07:32:53.333145 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.213766 32682 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.295373 31169 rpcz_store.cc:276] 0809 07:32:50.203753 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.142159 (+2938406us) service_pool.cc:227] Handling call
0809 07:32:53.210061 (+ 67902us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.294409 (+ 84348us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.294732 (+ 323us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:32:53.339525 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 168601us
I20260809 07:32:53.343781 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.334544 31175 raft_consensus.cc:2354] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:53.345137 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.323275 32269 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:53.349552 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.345667 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.195667 32566 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.338578 32397 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.339116 32662 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:53.336712 32657 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
W20260809 07:32:53.368597 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 198624us
I20260809 07:32:53.369602 31006 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:53.300661 32562 consensus_queue.cc:1159] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12107e38eb6d424abc4f45f16a9a1560" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260809 07:32:53.372610 31182 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.372365 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:53.297580 31164 rpcz_store.cc:276] 0809 07:32:50.297863 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.147337 (+2849474us) service_pool.cc:227] Handling call
0809 07:32:53.214547 (+ 67210us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:53.214889 (+ 342us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.260185 (+ 45296us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.260549 (+ 364us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":21184768}
I20260809 07:32:53.340785 32653 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260809 07:32:53.365271 32638 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.373796 31172 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.399032 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:53.393683 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 77331.9us
I20260809 07:32:53.379429 32486 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:53.383204 32549 consensus_queue.cc:1046] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260809 07:32:53.378484 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.402947 30853 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:32:53.406450 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 50454.9us
I20260809 07:32:53.383108 32599 raft_consensus.cc:2741] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:53.359102 32654 consensus_queue.cc:1046] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260809 07:32:53.355544 32269 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.382951 32566 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.415246 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.390107 32657 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.419993 32605 raft_consensus.cc:3032] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.420262 31045 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.373543 32481 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260809 07:32:53.410982 32717 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.369835 32703 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.412555 32661 consensus_queue.cc:1046] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260809 07:32:53.430948 32714 consensus_queue.cc:1046] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.432789 32556 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.460273 32556 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:53.460615 32556 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:53.351675 32511 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.416293 32476 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.461376 32556 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.421146 32721 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.471031 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:53.472669 32657 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.433501 32724 consensus_queue.cc:1046] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:53.458092 32461 raft_consensus.cc:3032] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.479681 32461 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.480520 32461 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.482548 32461 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.492254 32738 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.493227 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 601) took 1299 ms. Trace:
I20260809 07:32:53.495035 32721 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.495525 32661 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.476047 32617 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.495913 32661 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.462626 32511 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.430305 32644 consensus_queue.cc:1046] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260809 07:32:53.470695 32485 raft_consensus.cc:3032] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.411674 30896 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:53.425752 32605 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.426364 32599 consensus_queue.cc:1046] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260809 07:32:53.477466 32269 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.479274 30890 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:53.458539 32654 raft_consensus.cc:3032] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.544261 32654 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.545029 32644 raft_consensus.cc:3032] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.545460 32644 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:53.545212 32654 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.412606 32486 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.481598 32609 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:53.471334 32668 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.437354 32596 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260809 07:32:53.391855 32716 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260809 07:32:53.434918 32733 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.490919 32556 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.497404 31174 raft_consensus.cc:1226] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.499334 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 466) took 3198 ms. Trace:
I20260809 07:32:53.493607 31175 rpcz_store.cc:276] 0809 07:32:52.194196 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:52.843325 (+649129us) service_pool.cc:227] Handling call
0809 07:32:53.083973 (+240648us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99b7c 00007f4091f9973b 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:53.492941 (+408968us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400016dd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:53.493165 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":391429376}
W20260809 07:32:53.530733 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 115552us
I20260809 07:32:53.612558 31182 rpcz_store.cc:276] 0809 07:32:50.301125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.149328 (+2848203us) service_pool.cc:227] Handling call
0809 07:32:53.218532 (+ 69204us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:53.218703 (+ 171us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.374959 (+156256us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.499015 (+124056us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54000172d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:53.499257 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":389636736}
I20260809 07:32:53.614269 30853 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.531534 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:53.530547 32485 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.615164 30900 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:53.615761 32485 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.616592 32668 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.437078 32670 consensus_queue.cc:1046] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260809 07:32:53.542394 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:32:53.617816 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 75533.2us
I20260809 07:32:53.587829 32654 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.595772 32721 raft_consensus.cc:3032] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.496754 32661 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.598354 32617 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.563701 32644 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.534070 32682 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:53.481626 32703 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:32:53.608683 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 176257us
I20260809 07:32:53.619385 30900 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.608588 32609 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.643574 32721 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:53.605162 32579 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:32:53.609210 32715 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.112s user 0.003s sys 0.005s
I20260809 07:32:53.646610 30853 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.644747 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:53.616827 32485 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.570554 32452 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260809 07:32:53.667394 30853 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.672045 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.672773 31177 raft_consensus.cc:2379] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:53.665480 32721 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.670181 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:53.648267 32547 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:53.679049 30853 leader_election.cc:398] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:53.649964 32511 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:53.653388 32609 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.674125 32741 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.709352 32738 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.707069 32669 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260809 07:32:53.711742 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.709451 32736 raft_consensus.cc:3032] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.713722 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.732842 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.735041 31177 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.665649 32717 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:53.645413 32546 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 07:32:53.736593 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:53.674417 32481 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:53.710047 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.668277 32661 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.741724 31046 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.712417 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.741844 32717 raft_consensus.cc:3032] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.744807 30853 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.748862 31046 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:32:53.746215 32644 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.757727 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.745857 32669 consensus_queue.cc:1225] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260809 07:32:53.739943 32481 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:53.829558 32481 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.832782 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.732164 32547 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.732368 32511 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.749096 32717 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:53.737509 32721 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.745487 32461 raft_consensus.cc:3032] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.742617 32741 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:53.645390 32605 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.739034 32736 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.818295 32486 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.819461 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.820315 31059 raft_consensus.cc:1226] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:32:53.823129 31055 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.848778 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:53.823001 32609 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.822830 32662 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:53.824218 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.854626 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:53.855768 31049 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:53.854686 32662 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.856102 32609 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:53.851223 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:53.857599 31169 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:53.858613 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.824548 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.861835 30853 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.863296 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.824294 32738 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.838331 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.816339 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.824091 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.845842 31177 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:53.843116 32461 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.851269 30890 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:53.851553 31181 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.875972 32734 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.845742 32741 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.853127 31055 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260809 07:32:53.853928 31059 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.881274 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.864413 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 501) took 1081 ms. Trace:
I20260809 07:32:53.865134 30853 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.882347 31174 rpcz_store.cc:276] 0809 07:32:52.783042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.273213 (+490171us) service_pool.cc:227] Handling call
0809 07:32:53.497239 (+224026us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:53.612490 (+115251us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:53.864208 (+251718us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400016dd0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:53.864349 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":496452864}
I20260809 07:32:53.866304 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:53.868101 32733 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:53.867798 32547 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:53.870744 32609 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.873641 31164 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:53.873739 32397 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.874049 32461 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.874596 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.845233 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:53.852599 31057 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:53.880081 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.880574 31055 pending_rounds.cc:76] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 1
I20260809 07:32:53.837167 32511 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.885898 30853 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.886459 30890 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:53.886891 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:53.888906 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.882120 32741 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.888046 32452 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.899737 32461 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.901311 30898 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:53.885210 32733 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.890147 32397 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:53.913064 30853 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.914177 32734 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:53.916870 32716 raft_consensus.cc:3032] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:53.917552 32481 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.919917 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.920199 32605 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.910140 31055 pending_rounds.cc:97] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260809 07:32:53.941547 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.944912 31055 raft_consensus.cc:2866] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:32:53.945675 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.943315 32397 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.921788 32452 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.868726 32736 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.948524 30853 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.949775 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:53.950737 32739 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.127s user 0.001s sys 0.000s
I20260809 07:32:53.941491 32716 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:53.952139 32476 raft_consensus.cc:3032] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:53.952564 32476 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:53.952847 32562 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.953537 32562 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:53.954025 32562 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.954432 32562 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.940322 32733 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.952648 32452 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.943437 32579 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:53.956020 30853 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.884713 32717 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.967125 32736 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:53.969002 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.969959 31055 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:53.969952 32610 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:53.970504 32609 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:53.970882 32609 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.971220 32609 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:53.955492 32734 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:53.953588 32476 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.970057 30853 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.961522 32579 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.941944 32511 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.973595 32731 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:53.975056 32343 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260809 07:32:53.974146 32397 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.989895 32638 raft_consensus.cc:3032] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:53.973871 32734 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:53.962785 32662 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:53.991030 32579 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:53.984148 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:53.992238 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:53.990764 32566 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.018810 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.011587 32734 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.025173 30853 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.010782 32476 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.011332 32638 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.026013 31054 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:54.024590 32731 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.028007 30904 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:54.029036 30853 leader_election.cc:398] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:54.020547 32562 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.029865 32452 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.033224 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.047852 30853 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.063024 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:53.979374 32758 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.086s
I20260809 07:32:54.030714 31045 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:54.032923 32566 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.038717 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.073369 30853 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.036374 32511 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.074939 30853 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.070107 32596 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:32:54.075783 32605 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.091614 32511 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.092742 30853 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.093560 32511 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.093941 30853 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.069420 32343 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.073951 30890 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.095599 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.102614 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.128520 31054 raft_consensus.cc:2396] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:54.103447 32750 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.105792 32716 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:54.106684 303 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.089s user 0.001s sys 0.000s
I20260809 07:32:54.107977 32562 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:54.109484 32757 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.096s user 0.002s sys 0.000s
I20260809 07:32:54.072228 32609 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.076913 32481 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:53.900115 32546 consensus_queue.cc:1159] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c9de56c3fa3487c9dac990675c34be5" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260809 07:32:54.140956 32609 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.142486 32452 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.110841 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:54.115945 32755 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.143s user 0.000s sys 0.007s
I20260809 07:32:54.094532 32638 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.123610 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.125726 31057 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:54.126811 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.127346 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.126736 31059 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
W20260809 07:32:54.095273 32756 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.146s user 0.001s sys 0.001s
I20260809 07:32:54.130494 31054 raft_consensus.cc:1226] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:54.132464 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:54.144830 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 49270.5us
I20260809 07:32:54.147722 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.146580 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.155031 30853 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.150449 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.152010 31059 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:54.154155 31055 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:54.150153 32566 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.158584 30852 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.141752 32733 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.143079 32343 consensus_queue.cc:1046] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260809 07:32:54.163717 31045 raft_consensus.cc:2379] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:54.143189 32741 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.167701 32741 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.168207 30890 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:54.170493 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.172904 32397 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.173830 32397 consensus_queue.cc:1046] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.174639 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.175273 31182 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:54.172091 31057 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:54.135461 32750 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.141773 32662 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.094131 32734 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.178858 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.179020 32511 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.179147 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.183519 31174 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.183796 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.183969 31170 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:54.180042 32635 consensus_queue.cc:1046] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260809 07:32:54.185094 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.185626 32397 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.185964 32452 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:54.186455 32452 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:54.186656 32734 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.187352 32343 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.187870 32734 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:54.187954 32343 raft_consensus.cc:686] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.186656 32661 raft_consensus.cc:3032] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:54.181836 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.189080 32661 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:54.189996 31054 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:54.190119 32661 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.190263 32731 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:54.188524 32734 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.186888 32452 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.193133 32717 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:54.194226 32661 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.195189 32638 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:54.163815 32579 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.192703 32734 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.189133 32343 consensus_queue.cc:237] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.201221 32733 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.203500 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.178920 32481 raft_consensus.cc:3032] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.199115 32579 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.204901 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:54.205201 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 50255.8us
I20260809 07:32:54.206847 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.206194 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.177898 31175 raft_consensus.cc:1226] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:32:54.191380 32731 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.215926 30853 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.219841 32733 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.220996 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.224241 31049 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:54.225257 30852 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.205884 32481 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:54.226457 32579 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:54.205673 32547 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.215690 30745 catalog_manager.cc:5495] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:54.227083 32579 raft_consensus.cc:686] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.227048 32481 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.228375 32481 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.228257 32579 consensus_queue.cc:237] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.176831 32716 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.218541 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.228325 30745 catalog_manager.cc:5495] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:54.231396 31174 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:54.227023 32452 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.238615 32546 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.233826 32721 consensus_queue.cc:1046] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.239603 32721 raft_consensus.cc:3032] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:54.239959 32721 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.239995 32546 consensus_queue.cc:1046] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.241129 32397 raft_consensus.cc:3032] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.241439 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.241503 32397 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:54.241227 32721 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.242225 32546 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.183085 32750 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.237272 32731 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.242578 32721 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:54.244767 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.243641 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.246543 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.244800 32750 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.247277 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:54.245816 32731 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.233959 32511 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.249714 31006 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.252775 30745 catalog_manager.cc:5495] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:54.260464 30853 leader_election.cc:398] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:54.261075 30853 leader_election.cc:401] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:32:54.251710 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.242758 32397 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.259526 32481 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.268723 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.273500 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.274677 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.275275 31182 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:54.278985 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.250746 32638 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:54.264112 32736 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.261555 32714 consensus_queue.cc:1046] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 07:32:54.259851 32661 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.263690 32461 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:54.262815 32546 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.287035 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.287253 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.287618 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.288417 30911 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:54.288463 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.289477 30899 raft_consensus.cc:2396] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:54.289965 30911 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:54.281440 32684 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.261102 32476 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:54.280385 31174 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:54.280241 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.281096 31048 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:54.281354 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.278194 30745 catalog_manager.cc:5495] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:54.263768 31049 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:54.283082 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.285943 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.286690 32609 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:54.262619 32511 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 07:32:54.287621 30909 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:54.326694 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.327816 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.328491 30909 raft_consensus.cc:2362] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2. Re-sending same reply.
I20260809 07:32:54.329483 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.291194 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.330699 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.331338 31181 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:54.332144 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.332437 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.333069 31177 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:54.334132 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.280973 32638 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.297397 30904 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:54.271574 32452 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.280942 32481 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:54.279914 32635 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.326071 32609 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.338824 30745 catalog_manager.cc:5495] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:54.343262 31181 raft_consensus.cc:2379] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.281534 32547 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260809 07:32:54.343909 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.344655 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.292054 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:54.359513 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 78231.4us
I20260809 07:32:54.286778 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.345266 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.345697 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.360319 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.347816 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:32:54.348341 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 62409.2us
I20260809 07:32:54.343773 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.357555 30900 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.333659 32397 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.357749 30745 catalog_manager.cc:5495] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:54.344998 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.360423 31182 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:54.360648 30904 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:54.346264 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.333091 30898 raft_consensus.cc:2362] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2. Re-sending same reply.
I20260809 07:32:54.287284 32546 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.387286 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.341581 32638 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.385378 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.376423 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.394205 31182 raft_consensus.cc:2379] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.394740 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.377058 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.395980 31181 raft_consensus.cc:2379] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.394232 319 consensus_queue.cc:1046] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 07:32:54.397221 319 raft_consensus.cc:3032] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:54.397568 319 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.397758 32734 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:54.386397 31174 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.387292 31006 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.388052 30906 raft_consensus.cc:2362] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2. Re-sending same reply.
W20260809 07:32:54.444343 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 158383us
I20260809 07:32:54.389642 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.361351 32657 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:54.341693 32476 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.360109 30911 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:54.361646 31177 raft_consensus.cc:2379] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.391182 30745 catalog_manager.cc:5495] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:32:54.446085 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 58872.9us
I20260809 07:32:54.447566 30906 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:54.447019 30745 catalog_manager.cc:5495] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 2 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:54.392724 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.395008 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.395934 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.398183 31181 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:54.378898 30909 raft_consensus.cc:2362] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3. Re-sending same reply.
I20260809 07:32:54.449517 31182 raft_consensus.cc:2379] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.447441 31006 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.359257 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.408010 31049 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:54.385272 31170 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:54.411100 30898 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
W20260809 07:32:54.415079 311 log.cc:924] Time spent T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.070s user 0.001s sys 0.000s
I20260809 07:32:54.454417 32343 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 07:32:54.456122 32662 raft_consensus.cc:3032] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.456504 32662 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.458001 32662 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.459098 32662 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.419742 32609 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.421240 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.461091 31048 raft_consensus.cc:2379] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:54.423481 30896 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:54.422147 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.436323 31057 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:54.438052 32546 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:54.441705 31118 consensus_peers.cc:487] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 -> Peer 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299): Couldn't send request to peer 28b9965e21374d26b0ac2014c9bafdff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3bbfdffc27ac46a0a18582f85cd4a00a. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:32:54.464144 32556 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:54.452700 32717 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:54.465658 32717 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.466578 32717 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.441891 31169 raft_consensus.cc:1261] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:54.348609 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:54.465175 32556 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:54.470724 32556 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.397866 30853 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:32:54.445467 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 55924.3us
I20260809 07:32:54.472067 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:32:54.474373 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 76577.7us
I20260809 07:32:54.474365 30745 catalog_manager.cc:5495] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:54.476540 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.477311 32661 consensus_queue.cc:1046] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.478430 32661 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.479063 32661 raft_consensus.cc:3032] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:54.479429 32661 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:54.480434 32661 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.481850 32661 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:54.483314 32343 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.483553 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.484139 32461 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.400373 319 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.449630 31046 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:54.485335 319 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:54.452248 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.488504 32596 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:32:54.489699 30890 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:54.444694 30891 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:54.491611 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.450956 32657 raft_consensus.cc:686] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:54.454834 32476 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.448585 32481 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.448654 32635 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:32:54.386495 31172 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:54.484440 32547 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.454351 32736 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:32:54.475733 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.512178 30853 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.513761 30853 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.497867 30900 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:54.507577 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.448987 31005 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.528491 30853 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:32:54.528959 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 80018.8us
I20260809 07:32:54.529744 30853 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.515834 32731 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.518476 32638 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.519296 32657 consensus_queue.cc:237] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.520004 32397 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:32:54.521718 32736 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.492178 32609 raft_consensus.cc:2741] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:54.503099 32461 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.523306 32546 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.540275 32635 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.530261 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:54.533895 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.537153 30900 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:54.537770 304 raft_consensus.cc:2741] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:54.531402 32343 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.543614 32547 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:32:54.539384 32549 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.544656 32549 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.543534 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.545794 30900 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:54.546790 32549 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:54.530645 32721 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:54.547348 30853 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.536216 32703 consensus_queue.cc:1046] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260809 07:32:54.548169 32343 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.548422 30853 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.543040 32736 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.548791 32343 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.549258 32609 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.549577 32736 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.550161 32609 consensus_queue.cc:1046] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:54.550634 30853 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.551177 32609 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:54.551748 32511 raft_consensus.cc:3032] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.551923 32343 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.552206 32511 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.539933 32546 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.525831 32481 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.553565 32511 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.553951 32481 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.555613 32511 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.556805 32481 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:54.557345 32481 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.531148 32714 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.559311 32397 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.560046 32547 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.559851 32397 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.560544 32547 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.561391 32721 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.562192 32669 raft_consensus.cc:3032] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.562789 32669 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.562520 32714 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.563709 32669 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.565073 32669 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.565486 32547 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:54.565889 32546 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:32:54.540865 32635 raft_consensus.cc:686] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:54.560434 32717 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.542277 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.567638 32635 consensus_queue.cc:237] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.534634 32662 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260809 07:32:54.534409 32734 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.568014 32721 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.569440 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.558930 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.563939 32714 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.571122 30853 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.568397 320 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.582746 31054 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:54.583006 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.583748 31172 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:54.578166 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:54.582029 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.585109 31006 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:54.577293 304 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.585541 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:54.587522 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.587757 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.582010 32546 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:54.557812 32481 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.588671 32734 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:32:54.588867 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.567659 32717 raft_consensus.cc:686] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.595180 32750 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:54.596071 32741 raft_consensus.cc:2741] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:54.590435 32703 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.598531 32658 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:54.619520 32658 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:54.620015 32658 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.567256 32556 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.599802 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.600299 32547 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.628057 30853 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.603809 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.589380 32476 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.633030 31046 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:32:54.608569 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.607031 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.607990 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.634766 31055 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:54.606542 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.573058 32716 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260809 07:32:54.601681 32721 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260809 07:32:54.648475 328 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
I20260809 07:32:54.649680 32736 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.617750 32546 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:32:54.665024 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 58564.8us
I20260809 07:32:54.620605 32617 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.673244 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.633188 32750 raft_consensus.cc:686] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.675354 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:54.678467 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 80299.9us
I20260809 07:32:54.661355 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.601807 32670 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
I20260809 07:32:54.665150 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.682996 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:54.660619 32721 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:54.687266 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.667289 32644 raft_consensus.cc:3032] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:54.666530 32476 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:54.689505 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.635577 32547 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.668381 32741 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260809 07:32:54.707563 32481 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:54.602087 304 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.678910 32486 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:54.679157 32556 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.707525 32741 raft_consensus.cc:3032] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:54.683382 32617 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.697181 32644 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.721100 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.697818 31045 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:54.696255 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:32:54.694833 321 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.114s user 0.002s sys 0.000s
I20260809 07:32:54.712776 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.697440 318 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.697548 32721 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:54.698019 32596 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:54.750905 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:54.744406 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48246.4us
I20260809 07:32:54.721817 30909 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:54.719439 30899 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:54.753645 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:54.744223 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.751842 31005 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:54.717201 31175 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:54.757182 31177 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:54.758523 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:54.717001 32741 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.760883 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:54.747340 32461 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:54.749565 32617 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.808413 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:54.715602 32657 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260809 07:32:54.811045 30853 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.788369 320 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:54.749375 32397 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:54.721827 32638 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.754777 31170 raft_consensus.cc:1261] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:54.704363 32717 consensus_queue.cc:237] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.760113 318 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:32:54.838326 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 80457.6us
I20260809 07:32:54.788895 32684 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:54.838956 31046 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:54.760367 32714 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:54.813922 307 log.cc:924] Time spent T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260809 07:32:54.841604 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:54.807708 32703 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:54.887692 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:54.743794 32750 consensus_queue.cc:237] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.839598 32617 raft_consensus.cc:686] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.912974 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:54.749001 32546 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.753984 32644 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.846450 32744 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:54.895524 32397 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.892939 32684 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:54.835222 32461 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.833817 32731 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.063s
I20260809 07:32:54.894433 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.908577 32343 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:54.939394 32714 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.912598 320 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:54.928633 338 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:54.939965 32714 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:54.932890 32547 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:54.936292 32684 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:54.938030 32744 raft_consensus.cc:686] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:32:54.965231 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 76265.9us
I20260809 07:32:54.947435 30890 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:54.929488 32738 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260809 07:32:54.960783 330 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.278s user 0.001s sys 0.000s
I20260809 07:32:54.951278 32343 raft_consensus.cc:686] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:54.912364 32741 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:54.999835 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 112235us
I20260809 07:32:54.937461 32397 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.001255 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.967238 31175 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:54.963310 32703 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:55.007553 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:54.978605 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:54.967584 304 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:54.994087 32476 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 07:32:54.996158 32661 consensus_meta.cc:287] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.514s user 0.013s sys 0.005s
I20260809 07:32:55.023770 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.024433 31046 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
W20260809 07:32:55.018827 336 log.cc:924] Time spent T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.268s user 0.000s sys 0.001s
I20260809 07:32:54.961915 338 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.018743 32635 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:55.037544 31049 raft_consensus.cc:1261] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.037865 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.051957 30904 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:55.039707 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.041304 32397 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.041869 32461 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.042150 351 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:55.044461 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:55.045069 339 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.128s user 0.001s sys 0.000s
W20260809 07:32:55.034118 333 log.cc:924] Time spent T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.278s user 0.000s sys 0.001s
W20260809 07:32:55.044592 349 log.cc:924] Time spent T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.096s user 0.000s sys 0.001s
W20260809 07:32:55.061591 341 log.cc:924] Time spent T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.245s user 0.001s sys 0.000s
I20260809 07:32:55.054502 32758 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.062191 32658 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.063385 30899 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:55.049491 32635 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.081547 32486 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:55.093782 32635 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.094135 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:55.066494 337 log.cc:924] Time spent T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.217s user 0.000s sys 0.001s
I20260809 07:32:55.066829 32714 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.080008 31181 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:55.080760 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.054657 32744 consensus_queue.cc:237] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.002776 318 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.120815 32714 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:55.026360 32617 consensus_queue.cc:237] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.059535 32736 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:55.030618 320 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.097903 32658 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.099709 30890 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:55.080720 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.097376 32703 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.135598 31059 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
W20260809 07:32:55.147444 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 305914us
W20260809 07:32:55.129223 32669 consensus_meta.cc:287] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.564s user 0.007s sys 0.014s
I20260809 07:32:55.079053 32343 consensus_queue.cc:237] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.148918 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:55.158520 31182 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:55.176043 32714 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260809 07:32:55.178256 30899 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:55.179248 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.180119 30898 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:55.182740 30890 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
W20260809 07:32:55.173278 32511 consensus_meta.cc:287] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.617s user 0.021s sys 0.001s
I20260809 07:32:55.188400 30900 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:55.190672 32714 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:55.191228 32714 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.193630 32662 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:55.194583 32662 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.171512 32638 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
W20260809 07:32:55.203913 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 159552us
I20260809 07:32:55.210273 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:55.229202 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.229984 31182 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:55.230104 32738 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:55.238905 31182 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.237074 30853 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:55.237224 32461 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.248126 30906 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.228906 338 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.242261 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:55.228009 32638 raft_consensus.cc:686] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:55.240688 32476 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260809 07:32:55.239323 370 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.243357 353 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:55.256917 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.208949 32669 consensus_queue.cc:1046] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260809 07:32:55.235787 32733 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260809 07:32:55.271374 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.272532 31006 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:55.234992 32486 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260809 07:32:55.267179 353 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.250604 32461 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.297964 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:55.269724 32714 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.269528 362 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.117s
I20260809 07:32:55.283757 358 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:55.286087 353 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.363080 369 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:32:55.367170 32452 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:55.381083 32452 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:32:55.381373 375 log.cc:924] Time spent T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.107s user 0.000s sys 0.000s
I20260809 07:32:55.370524 338 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.382820 338 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.384692 338 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.385535 32635 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.386883 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:55.388173 364 log.cc:924] Time spent T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.153s user 0.002s sys 0.000s
I20260809 07:32:55.390522 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.391147 30911 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:55.379405 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.400916 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:55.401728 30900 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:32:55.388212 379 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260809 07:32:55.371433 32750 raft_consensus.cc:3032] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:55.361729 32669 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.371914 32662 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.371006 32717 raft_consensus.cc:3032] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:32:55.396616 32733 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.268680 370 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.380182 32461 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:55.424525 32721 consensus_meta.cc:287] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.638s user 0.005s sys 0.006s
I20260809 07:32:55.397091 32486 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.448040 32721 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.392289 32638 consensus_queue.cc:237] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.456176 30900 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:55.453819 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 96976.5us
I20260809 07:32:55.448207 32486 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.479904 370 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.481348 32486 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.482023 32635 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.440517 32750 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:55.477051 32721 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.478641 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:55.483309 32750 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.485030 32750 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.483315 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.488389 31182 raft_consensus.cc:2354] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.381843 32452 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.489415 31006 leader_election.cc:398] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.439909 366 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.493449 31006 leader_election.cc:401] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:32:55.491998 31181 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
W20260809 07:32:55.486186 32684 consensus_meta.cc:287] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.504s user 0.003s sys 0.003s
I20260809 07:32:55.496904 31006 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.446967 32717 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:55.470705 369 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.443465 32511 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260809 07:32:55.469766 32662 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.508738 32662 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.514281 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.515707 32717 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.506937 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.517385 32717 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:55.447350 32669 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.469219 32733 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.519914 32669 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.497880 366 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.520684 32733 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.522334 366 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.497350 32721 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.515911 31175 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:55.523556 353 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.527413 32758 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:55.518201 32662 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.504767 32486 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:32:55.525604 353 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.527815 369 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.533964 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:55.535496 32486 raft_consensus.cc:686] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:55.536512 32486 consensus_queue.cc:237] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.537459 369 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.499055 32684 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.544104 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.547204 31172 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.544819 31181 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:55.562832 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.541962 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.568962 319 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:55.572256 319 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.572661 319 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:55.585053 31181 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.573135 31175 raft_consensus.cc:2354] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.595948 32684 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.601998 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.602836 31169 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:55.603241 31006 leader_election.cc:398] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.603574 31006 leader_election.cc:401] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:32:55.604314 359 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.607223 31006 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.618834 31051 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.623533 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:55.627568 353 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.628079 353 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.630003 353 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.630911 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:55.631505 31165 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.634734 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.635367 31177 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:55.638659 31005 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:55.642242 31051 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:55.641528 31045 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:55.622052 388 ts_tablet_manager.cc:927] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Initiating tablet copy from peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.610075 32343 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.645656 31059 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:55.590914 369 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260809 07:32:55.616511 32617 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.648679 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.622350 32662 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:55.647571 32750 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:55.655481 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:55.661376 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.661367 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.668745 30904 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:55.662191 30909 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:55.699839 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.700706 30900 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:55.693523 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:55.652951 32662 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.702301 31165 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:55.700536 31048 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:55.650389 366 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:55.654986 370 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.707126 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:55.708711 32703 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.708385 370 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.709741 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.710117 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.710520 30911 raft_consensus.cc:2379] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:55.710728 30891 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:55.710590 370 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:55.713096 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.663075 32547 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:55.644178 32343 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.718120 30911 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260809 07:32:55.704319 363 log.cc:924] Time spent T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Append to log took a long time: real 0.051s user 0.000s sys 0.010s
I20260809 07:32:55.720479 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:55.726831 30904 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.728176 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.728897 30909 raft_consensus.cc:2354] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.652626 32617 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.730474 31048 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:55.732370 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:55.706297 32662 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.734500 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.731730 357 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.739567 30891 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:55.706146 32556 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.707971 344 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:55.718168 32547 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.755753 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:55.723133 32481 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:32:55.767287 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 49650.7us
I20260809 07:32:55.721019 32703 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.781561 30853 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:55.764174 32547 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.790184 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.796883 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.797550 31055 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:55.741928 32657 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260809 07:32:55.799682 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.805900 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.806569 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.806818 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:55.808503 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:55.809029 31055 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.807574 31054 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 07:32:55.809597 31054 pending_rounds.cc:76] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 1
I20260809 07:32:55.809831 31054 pending_rounds.cc:97] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:32:55.810050 31054 raft_consensus.cc:2866] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:32:55.809128 31058 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:55.811105 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.806566 31046 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:55.811774 31058 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:55.764662 368 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:55.812469 368 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.812973 368 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.800386 30911 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:55.819212 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.819442 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.822467 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.823073 31057 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.825862 30900 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:55.759645 32556 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.824846 30909 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:55.828248 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46784.2us
I20260809 07:32:55.831310 304 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:55.802501 338 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:55.835582 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.761454 344 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.837730 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.827906 32638 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.846295 31055 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:55.847200 30900 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:32:55.831056 396 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:55.829152 32670 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:55.802873 32731 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260809 07:32:55.852118 32562 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.852246 32684 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:55.843086 32661 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260809 07:32:55.812471 388 tablet_copy_client.cc:316] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Beginning tablet copy session from remote peer at address 127.28.87.67:44107
I20260809 07:32:55.853894 30891 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:55.807757 32566 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260809 07:32:55.872277 32562 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.854300 362 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:55.880764 32562 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.856564 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:55.881672 30909 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.882344 32741 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.831779 304 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.861673 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:55.863641 31170 raft_consensus.cc:1261] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.850035 338 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:55.831111 32658 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260809 07:32:55.891933 32556 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.856709 377 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:55.892674 377 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.893183 377 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.893723 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.896639 31175 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:55.859669 32670 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.896003 304 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.890710 30906 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:55.898964 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 44013.2us
I20260809 07:32:55.900628 31006 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.867266 32684 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.875667 32721 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.907276 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.907559 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:55.908358 31055 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:55.909380 32486 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.859067 32638 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.885293 351 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.918020 30909 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:55.927299 413 log.cc:924] Time spent T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20260809 07:32:55.902546 32670 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:55.937083 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.909932 32486 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.904572 32556 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260809 07:32:55.940907 32486 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.910784 32684 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:55.913232 387 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:55.911248 32343 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:55.913048 32617 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:55.965536 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 127869us
W20260809 07:32:55.969825 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 139010us
I20260809 07:32:55.970103 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:55.975221 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:55.976948 31059 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:55.942788 357 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:55.981267 32658 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:55.976810 32657 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:55.965936 387 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:55.945268 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:55.930950 351 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:55.971485 32343 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:55.984597 31172 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:55.978018 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.002394 32658 raft_consensus.cc:686] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:55.892264 338 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.009898 31177 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:55.995998 387 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:55.977363 32661 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260809 07:32:55.995381 426 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:56.019747 32461 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:32:56.033870 418 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
W20260809 07:32:56.035708 407 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260809 07:32:56.008185 359 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:55.996155 357 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 07:32:55.996201 32657 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.031258 426 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:56.053866 426 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:56.054503 32731 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:56.017851 344 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.057938 32731 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.010732 32343 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.067634 319 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.068912 32731 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.069761 31048 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:56.119946 319 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.120100 32717 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:56.120828 32717 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.121408 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.122207 31054 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:56.039269 32556 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260809 07:32:56.051038 32744 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260809 07:32:56.069629 344 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:56.175487 32731 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.186585 32731 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.187404 32579 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.188081 32579 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.189069 32731 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:56.158957 428 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.072s user 0.000s sys 0.002s
I20260809 07:32:56.196449 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.157881 359 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.201225 368 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.158804 32661 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.207677 32343 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.212508 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.216897 351 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.217074 31054 raft_consensus.cc:2379] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:56.117242 32658 consensus_queue.cc:237] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.217545 32661 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.230661 319 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.218403 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:56.234447 32609 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:56.245920 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 254661us
I20260809 07:32:56.231169 31170 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 07:32:56.242568 424 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.106s user 0.002s sys 0.000s
I20260809 07:32:56.194921 344 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:56.249274 420 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.233s user 0.001s sys 0.000s
I20260809 07:32:56.240260 32717 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.265097 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.269822 31054 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:56.279652 30852 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:56.219192 359 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.274333 31169 raft_consensus.cc:2354] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:56.266461 355 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:56.246910 319 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.287619 32579 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.296268 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.287570 32657 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.258203 32609 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.299120 32717 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.287140 32343 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:56.293672 32670 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.333724 32738 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.308084 32566 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.307734 32734 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.307804 32750 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.325109 32343 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:32:56.338068 32547 consensus_meta.cc:287] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.530s user 0.008s sys 0.003s
I20260809 07:32:56.320601 32744 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:56.335990 421 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.332077 32657 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:56.308290 355 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.356575 32731 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.331794 32717 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.352226 32343 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:32:56.370139 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 399643us
I20260809 07:32:56.334782 32738 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.372125 30853 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:56.327240 377 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.345104 32566 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.350699 32750 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.347563 32734 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.378979 31169 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:56.372864 32661 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.490406 355 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.491308 355 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.491767 355 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.371301 368 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:56.374645 32662 consensus_meta.cc:287] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.614s user 0.008s sys 0.004s
I20260809 07:32:56.358800 32744 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.356379 32547 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.359606 421 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:56.497759 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 125721us
I20260809 07:32:56.365887 32657 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.370081 32731 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:56.528105 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 263059us
I20260809 07:32:56.527369 421 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.524480 32609 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.530301 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.534451 355 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.535545 32734 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.538995 386 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.539762 32547 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.539454 386 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.510983 356 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.540571 356 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.541484 386 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.541997 386 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:56.542697 30898 raft_consensus.cc:2396] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:56.543188 356 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.544114 377 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.544703 319 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.534359 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.502951 344 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.508423 32741 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 07:32:56.555390 31057 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s user 0.014s sys 0.025s
W20260809 07:32:56.559689 31058 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s user 0.010s sys 0.047s
I20260809 07:32:56.547698 32731 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:56.560711 31057 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s user 0.014s sys 0.025s
W20260809 07:32:56.561021 31058 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s user 0.011s sys 0.047s
I20260809 07:32:56.571513 31169 raft_consensus.cc:1261] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260809 07:32:56.657814 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 148549us
I20260809 07:32:56.671844 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.671929 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.673695 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.674749 31169 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:56.676259 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.676442 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.677453 31170 raft_consensus.cc:2396] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:56.677608 31169 raft_consensus.cc:2396] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:56.553205 32638 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.375259 32684 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.620152 32744 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.686169 32717 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.689409 32684 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.689926 444 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.537171 436 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:32:56.573112 32670 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.614555 32703 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:56.579064 32758 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.573s user 0.037s sys 0.038s
I20260809 07:32:56.511384 359 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.529801 362 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.510849 368 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
W20260809 07:32:56.553894 369 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s user 0.000s sys 0.032s
I20260809 07:32:56.561792 377 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.612843 32657 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.617274 32738 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.684531 32579 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.743211 377 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.743803 32579 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.686141 338 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:32:56.575156 427 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.224s user 0.001s sys 0.000s
I20260809 07:32:56.751708 31182 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:56.753373 32734 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.754594 32738 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:56.755362 32738 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.560428 386 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.501230 32662 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:56.628060 426 consensus_meta.cc:287] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.573s user 0.007s sys 0.007s
I20260809 07:32:56.687839 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.694492 441 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:56.718209 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 45679.7us
I20260809 07:32:56.725039 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.725236 32741 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:56.733718 362 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.724924 32638 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.746801 31181 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:56.750787 32579 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.751667 338 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.754330 32734 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.759954 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:56.628610 440 log.cc:924] Time spent T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
I20260809 07:32:56.716943 32750 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.761106 426 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.759899 386 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.747934 377 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:56.772296 32758 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.762s user 0.037s sys 0.039s
I20260809 07:32:56.772581 30909 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:56.772804 461 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.785873 338 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.690798 444 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.755363 478 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.761384 441 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.796123 421 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:56.796938 421 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.798617 319 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:56.797035 32662 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.799582 319 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.791968 32734 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.798702 444 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.804742 31172 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:56.805665 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:56.792655 386 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:56.796967 387 consensus_meta.cc:287] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.724s user 0.002s sys 0.015s
I20260809 07:32:56.791574 478 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.814590 30909 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
W20260809 07:32:56.816694 409 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 1.076s user 0.012s sys 0.009s
I20260809 07:32:56.817270 426 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.800637 32638 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.823599 32638 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.823961 362 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.824536 32452 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:56.812280 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:56.818102 387 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.826498 31169 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:56.790151 461 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.832333 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.833365 31046 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:56.834551 31170 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:56.834399 461 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.793604 32741 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.837684 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:56.837604 482 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.839119 482 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.825471 421 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.844517 387 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:56.831970 474 log.cc:924] Time spent T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.156s user 0.000s sys 0.002s
I20260809 07:32:56.838690 30911 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:56.855753 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:56.855983 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:56.838543 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:56.860128 30896 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:56.860311 32343 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.878161 30906 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 5
I20260809 07:32:56.829823 386 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.839831 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:56.841276 478 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.898078 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:56.900170 30899 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:56.907660 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 561) took 1369 ms. Trace:
I20260809 07:32:56.908160 30906 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:32:56.908044 31182 rpcz_store.cc:276] 0809 07:32:55.538569 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.538677 (+ 108us) service_pool.cc:227] Handling call
0809 07:32:55.936919 (+398242us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:56.907407 (+970488us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b54003890d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:56.907586 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1739924224}
I20260809 07:32:56.910877 357 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.931597 444 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:56.932101 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.932264 444 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.932976 31058 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:56.933525 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:56.934340 31045 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:56.911571 357 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:56.931764 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.047930 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.201252 31055 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:57.094811 386 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.201798 31051 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:57.111228 31045 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:57.202514 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.955206 387 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.206162 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.049674 30909 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:56.986953 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 701) took 1199 ms. Trace:
I20260809 07:32:57.210441 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:56.987026 444 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.202886 32343 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:56.990739 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 708) took 1184 ms. Trace:
I20260809 07:32:56.991671 32684 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.220338 431 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.212944 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.220886 431 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.216053 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.222226 30890 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.220506 31059 rpcz_store.cc:276] 0809 07:32:55.806492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.806597 (+ 105us) service_pool.cc:227] Handling call
0809 07:32:56.990404 (+1183807us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540035c0d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:56.990565 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":245570304}
I20260809 07:32:57.224176 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:57.224975 31059 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:56.926672 32738 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:57.202790 31045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 726) took 1511 ms (client timeout 2004 ms). Trace:
W20260809 07:32:57.220108 437 log.cc:924] Time spent T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.685s user 0.007s sys 0.003s
I20260809 07:32:57.220997 32684 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.222976 30891 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:57.222365 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.178810 32486 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.208941 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:57.208580 31048 rpcz_store.cc:276] 0809 07:32:55.787637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.789927 (+ 2290us) service_pool.cc:227] Handling call
0809 07:32:56.986543 (+1196616us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54000db1d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:56.986764 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":628953984}
I20260809 07:32:57.228039 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:56.955391 357 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:56.991468 377 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:56.986012 426 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.233008 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.232949 30906 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:57.159948 32481 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.206928 32758 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.229518 387 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:57.226639 31045 rpcz_store.cc:269] 0809 07:32:55.691501 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:56.932995 (+1241494us) service_pool.cc:227] Handling call
0809 07:32:57.202724 (+269729us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:32:57.244920 30911 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.233356 32596 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:57.228500 30896 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:32:57.244050 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.233323 386 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260809 07:32:57.248996 31058 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.234704 444 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.250371 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.251694 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.253993 31048 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:57.258742 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.247877 319 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.261943 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.247285 357 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.262144 31059 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:57.266278 30904 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.259312 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.269908 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.272487 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.278484 32596 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.278990 32684 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.253800 32486 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.280604 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:57.286854 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 255710us
I20260809 07:32:57.253175 32758 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.262866 377 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.291571 431 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.260969 426 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.302480 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.264786 444 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.323261 31055 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:57.323518 31006 leader_election.cc:398] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:57.327410 31006 leader_election.cc:401] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260809 07:32:57.328583 384 log.cc:924] Time spent T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 1.627s user 0.009s sys 0.010s
I20260809 07:32:57.332960 436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:40216 (request call id 0) took 10348 ms. Trace:
I20260809 07:32:57.333367 436 rpcz_store.cc:276] 0809 07:32:46.921556 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:46.932204 (+ 10648us) service_pool.cc:227] Handling call
0809 07:32:47.135027 (+202823us) log.cc:842] Serialized 499 byte log entry
0809 07:32:47.372194 (+237167us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b1800332f80. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:32:57.265751 (+9893557us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":501,"mutex_wait_us":8479,"raft.queue_time_us":153167,"raft.run_cpu_time_us":2433,"raft.run_wall_time_us":118245,"spinlock_wait_cycles":3920000,"thread_start_us":263179,"threads_started":2,"wal-append.queue_time_us":29810,"wal-append.run_cpu_time_us":9624,"wal-append.run_wall_time_us":1018410}
I20260809 07:32:57.332653 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 579) took 1155 ms. Trace:
I20260809 07:32:57.331388 30890 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:57.340139 30890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 715) took 1535 ms (client timeout 1637 ms). Trace:
W20260809 07:32:57.340503 30890 rpcz_store.cc:269] 0809 07:32:55.804937 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.805596 (+ 659us) service_pool.cc:227] Handling call
0809 07:32:57.215818 (+1410222us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99c6c 00007f4091f99bd6 000000000034aea9 00007f4099dc5a30 00007f4099e8b7b4 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:57.340064 (+124246us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6311808}
I20260809 07:32:57.343060 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 660) took 2086 ms. Trace:
I20260809 07:32:57.343492 31174 rpcz_store.cc:276] 0809 07:32:55.256946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.257072 (+ 126us) service_pool.cc:227] Handling call
0809 07:32:55.270370 (+ 13298us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:32:55.270685 (+ 315us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:55.270711 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:32:55.734000 (+463289us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b180030f9a0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a
0809 07:32:56.798990 (+1064990us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b2c000425a8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:32:56.800082 (+ 1092us) log.cc:842] Serialized 52 byte log entry
0809 07:32:56.884843 (+ 84761us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:56.884874 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:56.897019 (+ 12145us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:57.342724 (+445705us) raft_consensus.cc:1671] finished
0809 07:32:57.342742 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:57.342893 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":46473,"raft.queue_time_us":3412,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":9548,"spinlock_wait_cycles":11977472,"thread_start_us":84280,"threads_started":1,"wal-append.queue_time_us":453588}
I20260809 07:32:57.336807 31175 rpcz_store.cc:276] 0809 07:32:56.176861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:56.183754 (+ 6893us) service_pool.cc:227] Handling call
0809 07:32:56.816948 (+633194us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:32:56.817283 (+ 335us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:56.832088 (+ 14805us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:56.832143 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:56.835406 (+ 3263us) log.cc:842] Serialized 26 byte log entry
0809 07:32:56.896856 (+ 61450us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:56.897064 (+ 208us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:56.906899 (+ 9835us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:57.330968 (+424069us) raft_consensus.cc:1671] finished
0809 07:32:57.330985 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:57.332480 (+ 1495us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":840,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":20509,"spinlock_wait_cycles":15956224,"thread_start_us":56641,"threads_started":1,"wal-append.queue_time_us":400409}
W20260809 07:32:57.346395 395 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.604s user 0.003s sys 0.008s
I20260809 07:32:57.339064 30911 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:57.352003 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 137792us
I20260809 07:32:57.264941 32481 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:57.356158 491 log.cc:924] Time spent T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.090s user 0.004s sys 0.000s
W20260809 07:32:57.354899 31005 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.622s (SENT) [suppressed 47 similar messages]
I20260809 07:32:57.358551 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 551) took 2176 ms. Trace:
I20260809 07:32:57.337222 357 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.360659 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.372169 32596 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.363756 31164 rpcz_store.cc:276] 0809 07:32:55.181723 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.183699 (+ 1976us) service_pool.cc:227] Handling call
0809 07:32:55.234752 (+ 51053us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:55.263131 (+ 28379us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:55.266545 (+ 3414us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:55.393225 (+126680us) log.cc:842] Serialized 26 byte log entry
0809 07:32:55.631839 (+238614us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:55.631877 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:55.632273 (+ 396us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:57.345484 (+1713211us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:57.347650 (+ 2166us) raft_consensus.cc:1671] finished
0809 07:32:57.347675 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:57.358356 (+ 10681us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3049,"raft.queue_time_us":4775,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":8660,"spinlock_wait_cycles":27948288,"thread_start_us":208953,"threads_started":1,"wal-append.queue_time_us":309210}
I20260809 07:32:57.364056 386 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:57.371412 32758 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.386564 32733 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:57.379875 387 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.388454 30890 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:57.395200 444 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.387135 32733 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:57.395910 493 log.cc:924] Time spent T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.058s user 0.000s sys 0.002s
I20260809 07:32:57.403111 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.405426 30911 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:57.413137 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.404017 386 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.407812 444 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:57.425299 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.409466 32486 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.411279 32481 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:57.428198 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 73344.2us
I20260809 07:32:57.413887 387 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260809 07:32:57.427026 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.431275 386 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:57.424379 304 consensus_meta.cc:287] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.470s user 0.004s sys 0.017s
I20260809 07:32:57.415769 436 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.433578 444 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.435463 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.426263 320 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.436347 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.438082 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.415802 32758 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.428803 30911 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.440623 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.456014 32758 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.456352 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.455925 30900 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:57.442102 32486 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.456959 31174 raft_consensus.cc:2354] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5.
W20260809 07:32:57.457263 497 log.cc:924] Time spent T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
I20260809 07:32:57.458654 444 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.459034 320 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.479945 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.480604 31164 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:57.481566 320 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.481751 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:57.460093 494 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.058s user 0.001s sys 0.000s
I20260809 07:32:57.460302 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 566) took 1733 ms. Trace:
I20260809 07:32:57.450520 304 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:57.479485 495 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.115s user 0.001s sys 0.000s
I20260809 07:32:57.484133 32758 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.484714 304 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.485617 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.487839 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.488427 31182 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:57.488682 441 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.489305 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.490043 31169 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:57.485532 30906 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:57.489100 30911 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:57.461808 30909 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:57.456313 436 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.478785 318 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:57.493448 318 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.482910 31165 rpcz_store.cc:276] 0809 07:32:55.726734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.726896 (+ 162us) service_pool.cc:227] Handling call
0809 07:32:55.727276 (+ 380us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:55.727686 (+ 410us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:55.727710 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:56.486448 (+758738us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b18001645c0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a
0809 07:32:56.992203 (+505755us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b2c000425a8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:32:56.992928 (+ 725us) log.cc:842] Serialized 26 byte log entry
0809 07:32:57.225676 (+232748us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:32:57.225763 (+ 87us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:57.230525 (+ 4762us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:57.459955 (+229430us) raft_consensus.cc:1671] finished
0809 07:32:57.459974 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:57.460137 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1631,"raft.queue_time_us":4409,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12805,"spinlock_wait_cycles":30089344,"thread_start_us":232376,"threads_started":1,"wal-append.queue_time_us":266553}
W20260809 07:32:57.462600 496 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.075s user 0.001s sys 0.000s
I20260809 07:32:57.442589 357 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.473659 31175 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:57.475001 500 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.476485 31005 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:57.473747 32661 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:32:57.495894 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 83625.4us
I20260809 07:32:57.478751 30900 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260809 07:32:57.479431 30898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 508) took 2293 ms (client timeout 2502 ms). Trace:
I20260809 07:32:57.491324 30904 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:57.494421 436 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.472370 32486 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.497825 32661 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.499315 377 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.500864 32566 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.477183 32644 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.519003 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.520411 31120 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.497249 30890 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:57.521910 444 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:57.522357 444 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.513912 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.514698 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.501276 500 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.515489 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.515934 31058 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:32:57.501575 30898 rpcz_store.cc:269] 0809 07:32:55.186339 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.189128 (+ 2789us) service_pool.cc:227] Handling call
0809 07:32:55.533823 (+344695us) spinlock_profiling.cc:243] Waited 8.99 ms on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99b7c 00007f4091f9973b 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:32:57.478979 (+1945156us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b540045f6d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:57.479197 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2236441216}
I20260809 07:32:57.526947 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.526525 444 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.529683 31164 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:57.531968 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.532825 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.532954 441 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.533699 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.533551 441 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.534373 436 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.534744 436 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.535079 436 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.535610 318 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.535633 441 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.536276 318 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:57.537139 32721 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:57.537248 30904 raft_consensus.cc:1261] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:57.537732 318 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.540627 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.529950 32609 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.543439 32596 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:32:57.549017 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.550336 30904 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:57.538138 320 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.539180 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.551565 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.543515 32758 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.553647 32758 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.554059 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.555549 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.556025 338 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.556483 338 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:57.556625 370 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:57.556910 338 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.557315 370 raft_consensus.cc:686] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:57.554021 32758 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.558538 370 consensus_queue.cc:237] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.558933 32617 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:57.561182 32635 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:57.542439 377 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:57.566354 31005 leader_election.cc:398] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:57.555624 500 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.568308 30899 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:32:57.569844 32486 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.570526 32609 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.571941 32733 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.572575 32609 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.572700 501 consensus_queue.cc:1035] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:32:57.573117 426 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.551788 320 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.568339 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.561729 32635 raft_consensus.cc:686] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:57.538250 318 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.564663 377 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.583323 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.565745 31175 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:57.585312 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.585469 30899 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:57.586212 32635 consensus_queue.cc:237] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.576200 32596 raft_consensus.cc:686] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:57.589922 30904 raft_consensus.cc:2354] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:57.577582 320 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.582378 377 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.593847 31054 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:57.590878 357 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:57.597062 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.602733 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.603098 482 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.600128 357 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.603991 357 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.604308 32596 consensus_queue.cc:237] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.597319 30890 raft_consensus.cc:1261] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:32:57.604822 338 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.599002 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.606860 30909 raft_consensus.cc:2396] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:57.607154 338 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.600728 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.601528 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.601778 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:57.602480 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 54372.6us
I20260809 07:32:57.608848 30900 raft_consensus.cc:2396] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:32:57.603417 30904 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:57.601348 32556 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.609038 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.603343 31045 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:57.612926 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:57.612730 32556 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.617576 426 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:57.618162 426 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:57.618610 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 744) took 1088 ms. Trace:
I20260809 07:32:57.619093 426 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.619982 436 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.621842 436 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.619004 31054 rpcz_store.cc:276] 0809 07:32:56.529781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:56.534050 (+ 4269us) service_pool.cc:227] Handling call
0809 07:32:57.618377 (+1084327us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000131d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:32:57.618507 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2391724288}
I20260809 07:32:57.623936 32661 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.626797 32758 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.628710 32758 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.629375 31006 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:57.631343 32486 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:57.632470 32556 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.636749 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.637714 32566 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:57.615834 318 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.637918 32717 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:57.638285 32566 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.638922 32717 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:57.639506 318 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:57.639878 318 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.640202 318 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.640980 32717 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.640363 32566 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:57.644133 448 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 1.054s user 0.002s sys 0.004s
I20260809 07:32:57.649266 30852 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:57.651669 377 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.653739 377 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.656266 387 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:57.657135 387 consensus_queue.cc:1225] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260809 07:32:57.657828 387 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:57.658334 387 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:57.659495 387 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.659456 320 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.632591 357 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.662264 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.662907 320 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:57.663363 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.664104 30900 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:57.662904 30906 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:57.669579 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.670581 30906 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:57.670573 387 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.673594 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.668649 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.669909 31006 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:57.676311 30900 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:57.676790 320 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.677220 320 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.677582 320 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.678999 32566 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.679382 32566 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.679697 32566 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.667431 444 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.680145 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.844254 30909 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:57.688683 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:57.892094 30900 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:57.889211 30911 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:57.615890 338 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:57.891780 30906 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:57.763787 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:57.681707 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46322.9us
I20260809 07:32:57.722215 510 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:57.898907 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40566 (request call id 150) took 4474 ms. Trace:
I20260809 07:32:57.636298 386 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.262s
I20260809 07:32:57.906863 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.907644 30906 raft_consensus.cc:2354] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:57.682476 357 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:57.702654 32758 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:57.882999 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:57.908535 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 144847us
I20260809 07:32:57.909111 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.911334 32721 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:57.765108 30744 catalog_manager.cc:5495] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3, now has a pending config: VOTER 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65), VOTER a720817fc113446a8ed55e61523b280a (127.28.87.66), VOTER 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67), NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:32:57.899255 510 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.821291 32609 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:57.917541 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.651669 318 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.767658 32566 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.921769 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.931000 30906 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:57.896888 338 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.932896 31174 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:57.914354 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.907562 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.918299 32721 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:57.939803 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.939350 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:57.941880 31174 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:57.940337 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.922562 32758 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.935366 338 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.956221 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.956986 31164 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:57.955765 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.961544 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:57.964514 31165 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:57.942741 426 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:57.967530 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.943694 32721 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:57.938957 370 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260809 07:32:57.942427 30898 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:57.942273 30890 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:57.972440 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.975435 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.978544 31181 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:57.948669 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.974643 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:57.976517 30852 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:57.938259 32609 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:57.979996 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.983191 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.987255 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:57.987581 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:57.985402 31174 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:57.977701 32661 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:57.979975 426 raft_consensus.cc:686] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:58.002274 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.004756 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.004930 32703 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.005451 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.005059 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.006570 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.007156 31057 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:57.992385 501 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.007838 501 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.008193 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.008220 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.005853 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.011658 31046 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:57.984810 32644 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:58.013384 32644 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.007778 319 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:58.014458 319 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.015774 31165 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:58.016417 31058 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:57.986204 32486 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:58.015513 32644 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.007064 351 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:58.005947 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:57.974100 320 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.022058 31054 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.010115 501 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.024586 319 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.028509 320 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.029111 31051 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:58.029278 32758 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.023217 31055 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:57.994831 441 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:58.030010 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.033521 31058 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.032698 31054 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.020773 31045 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:58.042742 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:57.902179 30745 rpcz_store.cc:276] 0809 07:32:53.357446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:53.367069 (+ 9623us) service_pool.cc:227] Handling call
0809 07:32:57.814710 (+4447641us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:32:54.646770 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:32:54.647527 (+ 757us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:32:54.647604 (+ 77us) write_op.cc:435] Acquired schema lock
0809 07:32:54.647652 (+ 48us) tablet.cc:660] Decoding operations
0809 07:32:54.649252 (+ 1600us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:32:54.649364 (+ 112us) write_op.cc:641] Partition lock acquired for write op
0809 07:32:54.649392 (+ 28us) tablet.cc:683] Acquiring locks for 9 operations
0809 07:32:54.673699 (+ 24307us) tablet.cc:699] Row locks acquired
0809 07:32:54.675326 (+ 1627us) write_op.cc:260] PREPARE: finished
0809 07:32:54.688507 (+ 13181us) write_op.cc:270] Start()
0809 07:32:54.698174 (+ 9667us) write_op.cc:276] Timestamp: P: 1786260774684933 usec, L: 0
0809 07:32:54.698198 (+ 24us) op_driver.cc:331] REPLICATION: starting
0809 07:32:54.761502 (+ 63304us) log.cc:842] Serialized 3618 byte log entry
0809 07:32:55.356760 (+595258us) op_driver.cc:444] REPLICATION: finished
0809 07:32:55.574767 (+218007us) write_op.cc:301] APPLY: starting
0809 07:32:55.574856 (+ 89us) tablet.cc:1365] starting BulkCheckPresence
0809 07:32:55.575103 (+ 247us) tablet.cc:1368] finished BulkCheckPresence
0809 07:32:55.588472 (+ 13369us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:32:55.594276 (+ 5804us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:32:55.594522 (+ 246us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0809 07:32:55.594576 (+ 54us) write_op.cc:312] APPLY: finished
0809 07:32:55.644859 (+ 50283us) log.cc:842] Serialized 70 byte log entry
0809 07:32:55.652372 (+ 7513us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:55.690266 (+ 37894us) write_op.cc:454] Released schema lock
0809 07:32:55.701507 (+ 11241us) write_op.cc:341] FINISH: Updating metrics
0809 07:32:55.721402 (+ 19895us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:55.724027 (+ 2625us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":217736,"apply.run_cpu_time_us":9605,"apply.run_wall_time_us":153140,"num_ops":9,"prepare.queue_time_us":57707,"prepare.run_cpu_time_us":22734,"prepare.run_wall_time_us":388036,"raft.queue_time_us":175650,"raft.run_cpu_time_us":21559,"raft.run_wall_time_us":137627,"replication_time_us":657496,"thread_start_us":464560,"threads_started":4,"wal-append.queue_time_us":258641,"wal-append.run_cpu_time_us":37656,"wal-append.run_wall_time_us":1648125}]]}
I20260809 07:32:58.051303 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.020218 351 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.055414 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.056171 31165 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:58.030894 32644 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.063634 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.064316 30911 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:58.033965 441 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.062431 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.038631 32661 raft_consensus.cc:686] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:58.005522 32703 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.111954 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.118594 510 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.120354 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.121693 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.122509 426 consensus_queue.cc:237] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.057646 318 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.128955 32609 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.110900 32566 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.071825 441 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.130926 32661 consensus_queue.cc:237] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.059849 351 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.142609 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.143654 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.107558 32617 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260809 07:32:58.142098 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.135221 31057 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:32:58.113658 32644 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.104451 444 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.111723 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.137717 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.138700 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.139730 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:58.140902 516 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.170s user 0.000s sys 0.000s
I20260809 07:32:58.141338 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.147711 31054 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:58.162420 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.163160 31164 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:58.153118 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:58.158043 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 181600us
I20260809 07:32:58.167941 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.178755 30904 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.154856 30744 catalog_manager.cc:5495] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 2, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 2 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:32:58.223343 513 log.cc:924] Time spent T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.456s user 0.015s sys 0.000s
I20260809 07:32:58.226445 30852 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:58.227319 526 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260809 07:32:58.227524 32609 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:58.227963 32609 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.228302 32609 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.229329 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.162305 30898 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:58.305241 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.358621 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:58.396972 31051 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:58.397276 31174 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:58.163308 31172 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:58.263043 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:58.227342 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 48611.1us
I20260809 07:32:58.312783 30904 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:58.402671 31055 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
W20260809 07:32:58.276662 520 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.222s user 0.000s sys 0.001s
I20260809 07:32:58.409680 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.412632 31051 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.407020 30744 catalog_manager.cc:5495] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:58.308048 444 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:58.421783 31055 raft_consensus.cc:1261] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:58.419447 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.343755 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.276836 32617 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.420063 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.423656 31058 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.426353 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.433763 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.438980 31051 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
W20260809 07:32:58.439615 387 consensus_meta.cc:287] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.768s user 0.009s sys 0.002s
I20260809 07:32:58.440193 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.440407 31059 raft_consensus.cc:2396] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:32:58.438346 31175 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:58.439855 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.441663 31051 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:58.441723 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.442521 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.442631 536 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:58.396299 32721 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.442865 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.416517 31045 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:32:58.440862 31058 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:58.539994 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.487798 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:58.541740 31051 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.542119 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.547574 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.546041 31055 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:58.411257 377 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:58.446228 30744 catalog_manager.cc:5495] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:58.566006 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.566766 31058 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:58.569540 31048 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.567329 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.570479 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.571069 31058 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:58.566779 377 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.505367 535 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:58.572949 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.574554 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:58.576884 377 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.576431 30744 catalog_manager.cc:5495] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 2 to 3, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:58.577822 32566 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:58.577958 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.578260 32566 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.578713 32566 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:58.578482 30744 catalog_manager.cc:5495] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 2 to 3, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:58.580860 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.473832 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.570832 31055 raft_consensus.cc:2379] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.432946 32596 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:58.541908 32644 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.465184 32703 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.583892 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.566417 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.598506 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.599278 31055 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.430536 444 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.505329 32609 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.476148 338 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.433599 32617 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.605623 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.615501 31055 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.613897 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.618346 31051 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:32:58.600073 32733 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:58.595937 32661 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:58.596156 31049 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.633863 31055 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:58.566550 536 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.637054 31182 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:32:58.615103 357 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:58.587805 535 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.638779 31054 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
I20260809 07:32:58.625741 30898 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:32:58.640228 556 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260809 07:32:58.643105 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.651063 32721 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.651170 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.593070 32596 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.656232 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.629266 32733 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.656906 31051 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.643265 32661 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.660094 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.643934 357 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:32:58.680720 31057 consensus_meta.cc:287] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.522s user 0.007s sys 0.003s
I20260809 07:32:58.681171 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:58.681841 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 103972us
W20260809 07:32:58.684324 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 211661us
I20260809 07:32:58.685814 568 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:58.687011 568 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.687539 535 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.685809 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.688974 535 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:58.689328 535 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.672344 32733 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:58.692662 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.695351 31046 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.679180 357 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.689635 535 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:32:58.691058 441 consensus_meta.cc:287] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.560s user 0.009s sys 0.000s
I20260809 07:32:58.715513 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.715544 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.716312 30909 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.689535 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.726214 30852 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.701270 32566 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.745179 444 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.713488 441 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.749816 338 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.760102 569 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:58.760536 569 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.760867 569 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:58.761255 32566 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.739962 32609 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.766841 32596 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.766952 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.767659 30891 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.768077 32661 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.780326 31058 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:58.752008 535 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.785024 535 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.790699 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.791564 30904 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:58.792353 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.793161 30904 raft_consensus.cc:2396] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:32:58.793967 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.798419 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.810081 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.810809 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:58.811468 30899 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:58.811712 31169 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:58.812639 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.816802 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.818216 536 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:32:58.819067 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 68865.2us
I20260809 07:32:58.820230 568 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:58.801090 32733 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.822064 441 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.822877 30900 raft_consensus.cc:2354] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:58.823132 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.823799 32733 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.800720 30909 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:58.824890 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.820418 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:58.824294 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.826171 30900 raft_consensus.cc:2362] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3. Re-sending same reply.
I20260809 07:32:58.805627 31006 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.806797 318 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:58.827162 31046 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.827736 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.830233 32721 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:58.830667 32721 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.831007 32721 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:58.828497 482 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:58.834754 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:58.834806 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.835940 31165 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.836583 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.828333 318 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:58.839162 31164 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:32:58.837232 31181 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 1.
I20260809 07:32:58.841138 31051 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.842463 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.843093 30900 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.812532 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.829257 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.843722 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.837208 32717 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:58.776891 32617 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.845628 30904 raft_consensus.cc:2354] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:58.845717 31006 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:58.847342 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 119971us
I20260809 07:32:58.848198 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.848717 30909 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:58.846926 32717 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.836473 482 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:58.798171 30891 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.850666 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.866787 31165 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:58.863036 32717 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.868641 30899 raft_consensus.cc:2396] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:32:58.866683 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.870761 387 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:58.878511 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.871025 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.871218 32721 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.871528 32617 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.874380 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:32:58.863160 351 consensus_meta.cc:287] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.695s user 0.009s sys 0.000s
I20260809 07:32:58.835304 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.876372 30911 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.881940 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.901610 30900 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:32:58.865286 441 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260809 07:32:58.902513 31055 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.903285 31006 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:58.845700 32661 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.878135 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.849577 30898 raft_consensus.cc:2354] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5.
I20260809 07:32:58.906168 31054 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.906168 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.906890 30904 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:58.865154 512 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.909404 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:58.982836 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 73521.1us
W20260809 07:32:58.983026 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 61183us
I20260809 07:32:58.983671 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:58.910269 30909 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:58.911375 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.910707 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.874985 482 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:58.986214 370 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:58.986645 370 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.986979 370 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:58.990895 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:58.992540 32721 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.993166 30900 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:58.980710 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:58.998653 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.980348 31048 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:58.999379 31181 raft_consensus.cc:2379] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:59.000268 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.000802 32717 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.001274 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.001928 31181 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:58.993136 32721 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:59.002490 32721 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:58.977399 31058 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:58.979822 32644 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:58.994483 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.995814 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.003486 31174 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:32:59.003192 32644 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.005599 32644 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.997956 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:58.906591 32703 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:58.842968 318 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.006959 31059 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.008664 31055 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:59.010218 338 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:32:59.002837 32721 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.013392 31048 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:32:59.014135 569 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:59.015239 593 log.cc:924] Time spent T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Append to log took a long time: real 0.258s user 0.000s sys 0.002s
I20260809 07:32:59.016160 569 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.016839 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.017855 32717 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:58.982470 32733 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:59.018638 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.019125 32733 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.007591 512 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.022027 512 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.022691 32721 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.847052 510 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:58.911593 535 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:59.006621 32703 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:58.914793 387 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:59.009148 31120 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:58.912631 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:58.876961 357 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.028340 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:58.911864 31169 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:32:59.019860 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.041435 30896 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:59.042268 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.034686 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.036289 31174 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:59.036700 32596 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.030988 535 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:58.914295 32617 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.053766 535 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.054128 32617 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.055358 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.056828 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.057025 31164 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:59.057611 30899 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:59.027848 510 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:59.044108 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:58.914505 351 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.044109 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.046253 30911 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:32:59.030475 32703 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.050241 387 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.051281 512 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:59.035408 318 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.066083 30900 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:59.069692 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:59.078070 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.078910 30911 raft_consensus.cc:2354] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5.
I20260809 07:32:59.079084 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.079865 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.067054 31049 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.067003 31181 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:59.084973 31177 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:59.085039 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.086491 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.087203 30911 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:59.088666 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.066632 510 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.085243 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.070789 32617 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:59.096783 351 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.077414 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.095086 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.104269 31048 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.105577 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.106453 30911 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:59.108562 30900 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.088372 444 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.113394 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.096421 32644 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260809 07:32:59.088161 570 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.086115 577 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.112680 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.117861 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.090003 32758 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.120121 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.091118 32721 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.122563 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.122541 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.120462 31051 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.121166 351 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260809 07:32:59.110119 32617 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.126160 32644 consensus_queue.cc:1035] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:32:59.121218 568 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:59.126930 568 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.121003 32758 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.127693 32758 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.129541 32733 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.118817 444 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.120138 577 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:59.130664 444 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.130939 357 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.133877 568 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.134919 377 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:59.135881 377 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.136307 377 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.120361 570 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.138710 570 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.143213 32596 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:59.144906 602 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:59.145344 602 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:59.145651 602 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.127173 32617 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.127350 338 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.147253 338 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.147608 338 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.086256 426 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260809 07:32:59.152042 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.153324 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.153945 31165 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.155138 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.156863 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.158524 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.159973 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:59.135782 566 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.454s user 0.004s sys 0.000s
I20260809 07:32:59.158949 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.165686 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.166657 602 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.167268 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 561) took 1227 ms. Trace:
I20260809 07:32:59.167654 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.167276 351 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.168876 351 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.168305 31177 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:59.167517 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.166719 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.163110 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.171501 31165 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:59.169368 351 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.167847 30906 rpcz_store.cc:276] 0809 07:32:57.940972 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:57.941083 (+ 111us) service_pool.cc:227] Handling call
0809 07:32:57.961253 (+ 20170us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:32:58.180614 (+219361us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:32:58.180642 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:32:58.410861 (+230219us) log.cc:842] Serialized 26 byte log entry
0809 07:32:58.590773 (+179912us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:32:58.590807 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:58.590871 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:32:59.135246 (+544375us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:32:59.166978 (+ 31732us) raft_consensus.cc:1671] finished
0809 07:32:59.166999 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:32:59.167186 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1054,"raft.queue_time_us":11545,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8253,"spinlock_wait_cycles":12809472,"thread_start_us":312942,"threads_started":2,"wal-append.queue_time_us":213079}
I20260809 07:32:59.158597 30899 raft_consensus.cc:2379] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:32:59.183178 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:59.176714 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 79781.7us
I20260809 07:32:59.121093 32661 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:59.179335 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.180135 614 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:59.180279 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.158202 577 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.187285 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.168810 602 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.189092 31174 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.169610 32733 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:59.187577 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:32:59.193126 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 289914us
I20260809 07:32:59.194259 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.197293 30909 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.192394 31165 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.166082 32703 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.184141 535 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:32:59.199667 597 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:59.189647 32609 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:59.209695 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.219074 31048 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.225487 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.226630 31059 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.226708 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:59.203668 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.200367 32661 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.225504 31006 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.186455 614 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.219200 32721 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:59.194870 444 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.226948 602 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:59.225322 32733 raft_consensus.cc:686] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:59.250154 32717 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:32:59.225477 32609 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:59.268258 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:59.250041 31170 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.249121 31045 raft_consensus.cc:1226] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:32:59.274802 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.257083 32661 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.251488 31048 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:59.281844 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.209909 370 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.245970 512 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260809 07:32:59.267980 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:59.343778 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 164500us
W20260809 07:32:59.319557 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 51393.4us
I20260809 07:32:59.323086 31046 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:32:59.350595 31005 leader_election.cc:398] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:32:59.351411 31005 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:59.351646 32566 raft_consensus.cc:2741] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 07:32:59.351770 370 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:59.352382 596 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.302s user 0.001s sys 0.002s
I20260809 07:32:59.320237 630 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.355145 630 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.357352 630 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.358014 351 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.359462 31177 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.360651 351 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:32:59.363070 617 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260809 07:32:59.275177 619 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.274883 31006 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.260700 614 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.332723 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.349265 30909 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:59.371153 31165 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:32:59.282380 32609 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.279474 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.374269 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:59.378911 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:32:59.334024 32661 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
W20260809 07:32:59.374912 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 114866us
W20260809 07:32:59.381994 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 100225us
I20260809 07:32:59.381263 31005 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:59.430998 30900 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:59.283069 482 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.445888 32566 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:59.449337 32566 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.280324 602 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.431097 32644 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.452270 30904 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:32:59.228426 597 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.376060 32617 raft_consensus.cc:2741] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 07:32:59.372606 619 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.446857 30911 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260809 07:32:59.433341 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 205756us
I20260809 07:32:59.451880 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.374825 32717 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:59.454133 32703 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.428921 32721 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:59.381795 351 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260809 07:32:59.455577 31169 raft_consensus.cc:2379] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.468793 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.469403 30890 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.450163 482 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.464788 30909 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:59.466075 30911 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:59.443542 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.470608 31175 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.471897 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.472481 30911 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:59.467732 30900 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:59.473412 351 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.473657 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.474232 30911 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:32:59.473963 351 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.475447 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:32:59.476315 351 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:32:59.476919 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.477591 30911 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:59.478359 30852 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.478711 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.479692 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.480641 482 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:32:59.481038 482 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:59.481426 482 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.481248 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:32:59.482129 30911 raft_consensus.cc:2354] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:59.483299 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.455148 30899 raft_consensus.cc:1261] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:32:59.484736 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.488009 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.459120 32644 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:32:59.494455 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.462828 619 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.485775 32617 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:59.343955 627 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:32:59.498911 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 61222.1us
I20260809 07:32:59.500144 30853 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.502391 31120 leader_election.cc:398] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:59.483621 31196 tablet_copy_service.cc:140] P 12107e38eb6d424abc4f45f16a9a1560: Received BeginTabletCopySession request for tablet 3bbfdffc27ac46a0a18582f85cd4a00a from peer 28b9965e21374d26b0ac2014c9bafdff ({username='slave'} at 127.0.0.1:45380)
I20260809 07:32:59.485381 31170 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:59.490440 32717 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:32:59.490643 357 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.504314 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.501470 31177 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.494633 32644 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.512931 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.499521 370 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:59.498960 351 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:59.516258 426 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:59.521692 377 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.526531 31196 tablet_copy_service.cc:161] P 12107e38eb6d424abc4f45f16a9a1560: Beginning new tablet copy session on tablet 3bbfdffc27ac46a0a18582f85cd4a00a from peer 28b9965e21374d26b0ac2014c9bafdff at {username='slave'} at 127.0.0.1:45380: session id = 28b9965e21374d26b0ac2014c9bafdff-3bbfdffc27ac46a0a18582f85cd4a00a
I20260809 07:32:59.533314 31174 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
W20260809 07:32:59.530617 621 log.cc:924] Time spent T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.248s user 0.000s sys 0.001s
W20260809 07:32:59.531213 624 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260809 07:32:59.531792 482 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.501929 627 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:32:59.463193 32733 consensus_queue.cc:237] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.506887 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:32:59.519137 622 log.cc:924] Time spent T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.172s user 0.001s sys 0.000s
I20260809 07:32:59.507759 591 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:59.519050 32596 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:59.510978 357 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.556883 357 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.511234 444 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.569351 377 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.503245 32617 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.571890 32617 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.544118 627 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.577059 591 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:59.577978 591 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.515504 568 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.583863 568 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.586472 31181 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:32:59.538978 370 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.589139 370 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.591115 370 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.592160 32566 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.593506 597 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.598603 32596 raft_consensus.cc:686] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:59.599715 32596 consensus_queue.cc:237] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.607149 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 775) took 1035 ms. Trace:
I20260809 07:32:59.607683 31045 rpcz_store.cc:276] 0809 07:32:58.571522 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:59.136413 (+564891us) service_pool.cc:227] Handling call
0809 07:32:59.136839 (+ 426us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:32:59.318907 (+182068us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:32:59.606859 (+287952us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b54001bf3d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:32:59.607080 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":210804096}
I20260809 07:32:59.500173 510 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.547605 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.551532 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.558127 482 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.559046 602 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.609766 31059 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.560978 603 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:59.551481 536 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260809 07:32:59.611008 510 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.614665 31054 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.549211 426 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:59.616171 426 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:59.608434 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 113537us
I20260809 07:32:59.547094 31006 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:32:59.619796 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 72781.3us
I20260809 07:32:59.620687 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.621703 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.622480 32717 consensus_queue.cc:1046] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:32:59.623438 32717 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:59.623994 32717 raft_consensus.cc:686] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:59.624962 32717 consensus_queue.cc:237] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.626405 32721 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:32:59.626840 32721 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.627254 32721 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:32:59.629508 31059 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.632337 605 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:32:59.632844 605 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:32:59.633237 605 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.634598 512 raft_consensus.cc:3032] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:32:59.635187 512 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:32:59.637351 30853 leader_election.cc:398] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5.
I20260809 07:32:59.637754 30853 leader_election.cc:401] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:32:59.656994 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.657819 31165 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:32:59.658147 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.658560 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.657842 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.663448 31174 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:32:59.670111 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.671900 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.673002 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.673331 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.674532 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:32:59.675038 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 162150us
I20260809 07:32:59.675735 31120 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.679049 31174 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
W20260809 07:32:59.659456 32758 consensus_meta.cc:287] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.531s user 0.009s sys 0.001s
I20260809 07:32:59.684116 31120 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:32:59.662434 570 consensus_meta.cc:287] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.523s user 0.002s sys 0.007s
W20260809 07:32:59.685601 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 304389us
I20260809 07:32:59.689189 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:32:59.690127 31046 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260809 07:32:59.671101 338 consensus_meta.cc:287] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.523s user 0.000s sys 0.008s
I20260809 07:32:59.702713 31058 raft_consensus.cc:1261] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:32:59.698931 31048 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:32:59.704396 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:32:59.707846 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 70673.7us
I20260809 07:32:59.708556 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.671391 32566 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5.
I20260809 07:32:59.711233 31054 raft_consensus.cc:1261] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:32:59.712258 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.713593 31051 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.713801 31120 leader_election.cc:398] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5.
I20260809 07:32:59.714072 31120 leader_election.cc:401] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260809 07:32:59.717818 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.674710 318 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:32:59.706234 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.692282 32758 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.669808 577 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.701323 370 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:32:59.721550 646 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260809 07:32:59.705582 568 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.729795 568 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.730370 568 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.732481 614 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.715317 32617 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:32:59.713294 338 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.721710 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.701540 535 raft_consensus.cc:686] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:59.723800 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40582 (request call id 148) took 5170 ms. Trace:
I20260809 07:32:59.724028 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.668833 32661 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.747511 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:32:59.725047 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.750372 31054 raft_consensus.cc:2354] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4.
I20260809 07:32:59.636132 512 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.699581 30890 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:32:59.718948 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.721637 602 raft_consensus.cc:2741] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 07:32:59.724785 31057 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:32:59.676286 32609 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.723989 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.724171 30742 catalog_manager.cc:5495] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:59.724946 651 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:59.730274 31175 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:32:59.762260 30742 catalog_manager.cc:5495] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:59.725981 31045 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:32:59.738943 338 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.806324 30742 catalog_manager.cc:5495] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 4 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:32:59.812703 651 raft_consensus.cc:686] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:59.813751 651 consensus_queue.cc:237] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.818176 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.817961 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.822263 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.742411 30744 rpcz_store.cc:276] 0809 07:32:54.553196 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:54.557556 (+ 4360us) service_pool.cc:227] Handling call
0809 07:32:59.723719 (+5166163us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:32:58.726416 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:32:58.727260 (+ 844us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:32:58.727325 (+ 65us) write_op.cc:435] Acquired schema lock
0809 07:32:58.727346 (+ 21us) tablet.cc:660] Decoding operations
0809 07:32:58.728367 (+ 1021us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:32:58.728488 (+ 121us) write_op.cc:641] Partition lock acquired for write op
0809 07:32:58.728535 (+ 47us) tablet.cc:683] Acquiring locks for 6 operations
0809 07:32:58.729225 (+ 690us) tablet.cc:699] Row locks acquired
0809 07:32:58.729271 (+ 46us) write_op.cc:260] PREPARE: finished
0809 07:32:58.729524 (+ 253us) write_op.cc:270] Start()
0809 07:32:58.729668 (+ 144us) write_op.cc:276] Timestamp: P: 1786260778729378 usec, L: 0
0809 07:32:58.729707 (+ 39us) op_driver.cc:331] REPLICATION: starting
0809 07:32:58.730474 (+ 767us) log.cc:842] Serialized 2710 byte log entry
0809 07:32:59.183682 (+453208us) op_driver.cc:444] REPLICATION: finished
0809 07:32:59.453240 (+269558us) write_op.cc:301] APPLY: starting
0809 07:32:59.453375 (+ 135us) tablet.cc:1365] starting BulkCheckPresence
0809 07:32:59.513950 (+ 60575us) tablet.cc:1368] finished BulkCheckPresence
0809 07:32:59.531806 (+ 17856us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:32:59.581167 (+ 49361us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:32:59.581448 (+ 281us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0809 07:32:59.581508 (+ 60us) write_op.cc:312] APPLY: finished
0809 07:32:59.581837 (+ 329us) log.cc:842] Serialized 52 byte log entry
0809 07:32:59.582037 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:59.582139 (+ 102us) write_op.cc:454] Released schema lock
0809 07:32:59.582200 (+ 61us) write_op.cc:341] FINISH: Updating metrics
0809 07:32:59.594334 (+ 12134us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:32:59.594364 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":189549,"apply.run_cpu_time_us":4613,"apply.run_wall_time_us":153613,"num_ops":6,"prepare.queue_time_us":85907,"prepare.run_cpu_time_us":12164,"prepare.run_wall_time_us":30193,"raft.queue_time_us":134848,"raft.run_cpu_time_us":33041,"raft.run_wall_time_us":174837,"replication_time_us":447670,"spinlock_wait_cycles":1920,"thread_start_us":240690,"threads_started":4,"wal-append.queue_time_us":25791}]]}
I20260809 07:32:59.846657 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.847366 31054 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.838387 32617 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
W20260809 07:32:59.867210 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 106436us
I20260809 07:32:59.833335 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.728549 577 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.701006 570 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.728428 370 raft_consensus.cc:686] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:59.728046 32758 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.741605 318 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.761257 634 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:32:59.840368 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 147035us
I20260809 07:32:59.884938 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.842537 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:32:59.724716 619 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.887933 31048 raft_consensus.cc:2354] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:59.888994 619 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.819016 512 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.837965 32733 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:32:59.826496 351 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5.
W20260809 07:32:59.849539 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 131797us
W20260809 07:32:59.862675 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 138751us
I20260809 07:32:59.890756 32758 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:59.891247 30853 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.891821 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.892652 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.893297 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.828425 31169 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:59.808301 30742 catalog_manager.cc:5495] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:59.894302 637 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:32:59.894589 577 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.895231 577 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.895908 577 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.895370 30742 catalog_manager.cc:5495] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:32:59.893920 338 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260809 07:32:59.899048 338 raft_consensus.cc:3032] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:32:59.899482 338 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:32:59.846239 614 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.895212 32596 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.894021 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.900609 32733 raft_consensus.cc:2741] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:32:59.899467 597 raft_consensus.cc:2741] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 07:32:59.900954 32596 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.878115 30852 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.901044 338 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.902232 614 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.902678 614 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:32:59.903040 614 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.897928 30742 catalog_manager.cc:5495] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 3 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:59.904481 597 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:32:59.905035 597 raft_consensus.cc:686] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:32:59.906214 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.906129 597 consensus_queue.cc:237] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.901774 32596 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:32:59.907449 338 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:32:59.912334 614 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.914717 614 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.913964 31120 leader_election.cc:398] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:59.916333 31120 leader_election.cc:401] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260809 07:32:59.921406 30742 catalog_manager.cc:5495] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 2 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:32:59.921191 634 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.926370 570 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.926807 634 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:32:59.927407 357 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.929494 357 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.931923 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:32:59.932592 31172 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:32:59.866784 535 consensus_queue.cc:237] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:32:59.935062 661 log.cc:924] Time spent T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
W20260809 07:32:59.936779 654 log.cc:924] Time spent T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.072s user 0.000s sys 0.002s
I20260809 07:32:59.921123 30853 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.939615 31006 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.846903 32644 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.947618 30742 catalog_manager.cc:5495] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 3 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:59.964133 30742 catalog_manager.cc:5495] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 2, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:32:59.886005 32617 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:32:59.975817 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 54755.9us
I20260809 07:32:59.976660 30853 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.932485 31196 tablet_copy_source_session.cc:205] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Tablet Copy: opened 0 blocks and 1 log segments
W20260809 07:32:59.980360 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 61558.5us
I20260809 07:32:59.980700 31059 raft_consensus.cc:1226] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:32:59.961707 387 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.994517 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:32:59.995189 31057 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:32:59.967172 570 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:32:59.975677 619 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:32:59.974933 31175 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
W20260809 07:33:00.000263 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 115010us
I20260809 07:32:59.885881 627 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.001945 30852 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:32:59.979785 630 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:00.004201 30853 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.008208 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.009359 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:32:59.975047 357 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260809 07:32:59.996941 387 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:00.044328 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.044481 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.044340 31055 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:32:59.994719 32617 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.046891 30853 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:32:59.978976 32721 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:32:59.986266 577 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.003075 619 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:00.049644 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.063418 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.009370 630 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:00.065653 30904 raft_consensus.cc:1261] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:00.065773 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.002942 32661 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:32:59.995836 602 consensus_queue.cc:1046] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260809 07:33:00.036909 536 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.069552 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.196066 31051 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:00.207787 30909 raft_consensus.cc:2354] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5.
I20260809 07:33:00.048413 32609 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.009624 655 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.047997 569 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:00.043831 675 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:00.206399 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 160205us
I20260809 07:33:00.047904 651 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:00.072981 619 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.048364 387 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.076658 630 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.229056 32609 raft_consensus.cc:3032] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:00.229614 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.076718 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.047732 664 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.199s
I20260809 07:33:00.234438 30852 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:00.237418 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 193014us
I20260809 07:33:00.208881 535 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:00.240177 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 193363us
W20260809 07:33:00.249065 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 196598us
I20260809 07:33:00.007479 370 consensus_queue.cc:237] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.230825 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.251567 31005 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:00.232201 569 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:00.250150 32644 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.253450 577 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:00.255493 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbfdffc27ac46a0a18582f85cd4a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:33:00.255358 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.256520 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:00.258867 668 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.216s user 0.000s sys 0.000s
I20260809 07:33:00.259464 30899 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:00.251039 30898 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:00.260811 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.241259 32609 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:00.273381 535 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:00.304530 32609 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.305732 32609 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:00.262794 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.308593 30904 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.233191 318 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.269865 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.273538 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.249634 675 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.274230 30853 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.262919 627 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.276324 31054 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:00.288295 32721 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.288439 646 raft_consensus.cc:2741] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 07:33:00.290122 32596 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.292375 31005 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.295989 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.298415 351 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:00.299100 31170 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:00.284519 569 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.323320 31005 leader_election.cc:398] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:33:00.323573 351 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.324204 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:00.325305 31005 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.327417 32596 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:00.327921 31172 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.328781 318 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.329542 318 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:00.329895 318 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:00.330230 318 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.330420 32721 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 07:33:00.330783 568 consensus_meta.cc:287] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.600s user 0.010s sys 0.000s
I20260809 07:33:00.315873 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.285434 31049 raft_consensus.cc:2396] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:00.305044 535 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:00.331172 568 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:00.332465 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 58312.7us
I20260809 07:33:00.350332 30853 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.333601 31051 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:33:00.335923 387 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.336277 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:00.271183 651 raft_consensus.cc:686] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:00.342262 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 46375.1us
W20260809 07:33:00.342422 605 consensus_meta.cc:287] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.699s user 0.011s sys 0.000s
I20260809 07:33:00.363907 30853 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.352284 30904 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:00.384387 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:00.387900 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 51705.7us
I20260809 07:33:00.391280 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.393748 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.394744 603 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.387287 630 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.395685 568 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:00.395995 30906 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:00.396922 630 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.390578 700 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260809 07:33:00.398941 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.401105 30853 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.348205 426 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.349448 535 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.404964 30853 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.355893 30896 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.349578 634 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.412518 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:00.412990 31181 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:00.414417 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.413303 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.415161 31048 raft_consensus.cc:2379] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:00.415131 30853 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.415230 31058 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:00.417150 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.417485 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.402084 357 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.391064 698 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:33:00.419375 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.407209 627 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:00.349052 591 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260809 07:33:00.421068 31054 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:00.424340 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.420426 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.439585 31048 raft_consensus.cc:2379] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:00.454114 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.454442 32703 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:00.454917 32703 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.455667 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.456868 32703 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.456833 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.450326 318 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:00.420687 627 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:00.458206 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.458607 627 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.459447 318 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.460144 698 raft_consensus.cc:3032] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:33:00.460512 698 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:00.460721 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.465102 569 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.448892 387 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.468327 31058 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:33:00.469285 31120 leader_election.cc:398] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4.
I20260809 07:33:00.469583 31120 leader_election.cc:401] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260809 07:33:00.416044 426 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.058s
I20260809 07:33:00.421521 640 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.386008 605 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.444927 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:00.477639 31049 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:00.479882 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.409807 568 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:00.444825 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.403280 32609 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260809 07:33:00.494203 31058 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:00.448645 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.445216 32661 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260809 07:33:00.408735 619 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.449429 351 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:00.450071 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.421350 32596 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260809 07:33:00.436611 338 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260809 07:33:00.438822 377 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.448544 591 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.445564 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.453085 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.474388 31057 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.406440 630 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 07:33:00.476131 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 61070.5us
I20260809 07:33:00.500576 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.504683 31182 raft_consensus.cc:1226] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.505447 568 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.506646 31165 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.447537 577 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.508816 31120 leader_election.cc:398] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3.
I20260809 07:33:00.515969 510 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:00.504309 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.516321 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.515050 387 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260809 07:33:00.519021 632 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:00.493728 426 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4.
I20260809 07:33:00.514505 32661 consensus_queue.cc:1225] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260809 07:33:00.530210 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.568123 31181 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.521869 31045 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:00.515272 444 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260809 07:33:00.585089 31174 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.591506 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:00.592249 426 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:33:00.593031 426 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.593276 710 consensus_queue.cc:1046] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:00.594250 710 raft_consensus.cc:3032] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.594662 710 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:00.594347 426 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.568961 32721 consensus_queue.cc:1046] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260809 07:33:00.596028 32721 raft_consensus.cc:3032] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.596381 32721 raft_consensus.cc:729] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:00.597313 32721 consensus_queue.cc:260] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.599884 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.601462 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.606670 30904 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:00.606876 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.610773 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.515599 718 consensus_queue.cc:1046] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260809 07:33:00.614336 30900 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:00.615113 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:00.615420 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.616058 30900 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.623641 31059 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:00.624899 31059 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:00.619951 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.600867 31170 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:00.513952 651 consensus_queue.cc:237] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.637653 30853 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.638190 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.582685 569 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.632200 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.633082 30906 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
W20260809 07:33:00.611757 690 log.cc:924] Time spent T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.220s user 0.000s sys 0.001s
I20260809 07:33:00.637712 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.570628 510 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.577955 616 consensus_queue.cc:1046] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260809 07:33:00.648489 31057 raft_consensus.cc:2396] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:00.600405 627 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.587556 370 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260809 07:33:00.627575 32721 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.569756 591 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.061s
W20260809 07:33:00.653503 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 110936us
I20260809 07:33:00.653649 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.521790 713 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260809 07:33:00.647377 482 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:33:00.666849 30900 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:00.650622 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.667595 31057 pending_rounds.cc:76] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 1
I20260809 07:33:00.612246 646 consensus_queue.cc:1046] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260809 07:33:00.622766 32661 consensus_queue.cc:1046] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260809 07:33:00.671670 31058 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260809 07:33:00.692272 31057 pending_rounds.cc:97] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:33:00.697029 31058 pending_rounds.cc:76] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 1
W20260809 07:33:00.672086 31005 leader_election.cc:334] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.582s (SENT)
I20260809 07:33:00.673485 30853 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.639145 377 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:33:00.652472 664 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.693802 31120 leader_election.cc:398] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5.
I20260809 07:33:00.658082 32609 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:00.697345 30890 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:00.656980 570 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 07:33:00.701948 646 raft_consensus.cc:3032] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.702415 31051 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.697381 31058 pending_rounds.cc:97] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260809 07:33:00.710183 31058 raft_consensus.cc:2866] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:00.698292 31057 raft_consensus.cc:2866] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:00.537858 698 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:00.698580 31005 leader_election.cc:334] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.812s (SENT)
I20260809 07:33:00.692914 630 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.708250 30853 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.705287 31120 leader_election.cc:401] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260809 07:33:00.696066 605 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.715250 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.698220 426 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260809 07:33:00.716432 31005 leader_election.cc:334] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.823s (SENT)
I20260809 07:33:00.706578 30898 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:00.718705 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.709990 31175 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:00.715735 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.700446 616 raft_consensus.cc:3032] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:00.697729 30896 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:00.710517 619 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.721972 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.709837 482 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.714295 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:00.723486 31005 leader_election.cc:334] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.844s (SENT)
I20260809 07:33:00.717589 31174 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.725373 30853 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.709540 646 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:00.731545 648 raft_consensus.cc:3032] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.733206 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.722330 444 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5.
I20260809 07:33:00.740150 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:00.719429 32733 consensus_queue.cc:1159] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12107e38eb6d424abc4f45f16a9a1560" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260809 07:33:00.744881 31170 raft_consensus.cc:2379] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:00.745210 426 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:00.724745 616 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.726975 664 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260809 07:33:00.744084 31005 leader_election.cc:334] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 2.502s (SENT)
I20260809 07:33:00.747222 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.749125 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.746510 616 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:00.729487 597 consensus_queue.cc:1046] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260809 07:33:00.745095 627 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.730010 698 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:00.728971 632 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.739627 568 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:00.763691 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.728477 712 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.136s
I20260809 07:33:00.753381 31005 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.750378 634 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:00.747684 32661 consensus_queue.cc:1046] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260809 07:33:00.766525 570 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:33:00.767123 619 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:00.740849 648 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:00.750891 510 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.737076 640 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.786166 30853 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.786437 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.830022 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:00.787794 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 45032.8us
W20260809 07:33:00.831774 31005 leader_election.cc:334] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.909s (SENT)
I20260809 07:33:00.832002 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.832258 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.789680 31051 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:00.832885 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.804617 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.833895 30906 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.837455 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.838955 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.814273 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.804852 32596 raft_consensus.cc:3032] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:33:00.826538 31174 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:00.842651 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.841989 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:00.849057 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.854054 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.854856 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.855214 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.855641 30900 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:00.842757 426 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.856470 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.857342 31170 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:00.858439 444 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:00.817883 646 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.858925 444 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.859851 444 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.861693 388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59116 (request call id 2) took 5503 ms. Trace:
I20260809 07:33:00.782223 535 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.855873 30911 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:00.793879 32661 raft_consensus.cc:3032] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:00.864414 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.793835 664 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:00.875311 664 raft_consensus.cc:686] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.862087 388 rpcz_store.cc:276] 0809 07:32:55.362603 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:55.364687 (+ 2084us) service_pool.cc:227] Handling call
0809 07:32:55.648269 (+283582us) ts_tablet_manager.cc:928] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Initiating tablet copy from peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
0809 07:33:00.253396 (+4605127us) tablet_copy_client.cc:361] Tablet copy session begun
0809 07:33:00.457155 (+203759us) tablet_metadata.cc:641] Metadata flushed
0809 07:33:00.501580 (+ 44425us) tablet_copy_client.cc:490] Wrote new tablet metadata
0809 07:33:00.860530 (+358950us) tablet_copy_client.cc:411] Wrote new consensus metadata
0809 07:33:00.860975 (+ 445us) tablet_replica.cc:205] Creating consensus instance
0809 07:33:00.861434 (+ 459us) ts_tablet_manager.cc:983] Replica registered.
0809 07:33:00.861584 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":28276,"tablet-copy.queue_time_us":231613,"thread_start_us":262239,"threads_started":2}
I20260809 07:33:00.876268 664 consensus_queue.cc:237] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.877076 388 tablet_copy_client.cc:791] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Starting download of 0 data blocks...
I20260809 07:33:00.863286 31164 raft_consensus.cc:2396] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:00.846969 569 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260809 07:33:00.846647 32596 raft_consensus.cc:729] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.788115 597 raft_consensus.cc:3032] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.880128 32596 consensus_queue.cc:260] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.880700 597 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:00.799518 619 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.881354 32596 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:00.881790 31045 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:00.787122 712 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.882467 712 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:00.882838 619 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.793203 634 raft_consensus.cc:686] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:00.806155 318 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.883571 318 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:00.865814 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.884037 634 consensus_queue.cc:237] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.884923 318 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.866485 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.788539 30890 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:00.864046 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.782794 616 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.786500 650 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:00.870087 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.886610 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.858192 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.870375 32661 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.863502 30891 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.871393 31054 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:00.871658 535 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:00.872316 646 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.868856 648 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.874136 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.853776 577 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260809 07:33:00.902551 31172 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:00.903477 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.903759 31172 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 07:33:00.904094 31172 pending_rounds.cc:76] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 1
I20260809 07:33:00.904312 31172 pending_rounds.cc:97] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:33:00.904542 31172 raft_consensus.cc:2866] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:00.905916 31172 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:00.875226 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:00.907078 31048 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:00.834779 699 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.812623 710 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.909825 710 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.909945 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.867534 627 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.888202 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.915961 30904 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:00.855418 377 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.889918 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.920274 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.921075 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.869908 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.890440 30906 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:00.922969 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.923709 30906 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:00.841804 338 raft_consensus.cc:3032] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:00.834513 570 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:00.891701 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:00.916780 650 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:00.940253 650 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.940779 30891 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:00.941759 31164 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:00.843633 675 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:00.844347 32609 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:00.881588 597 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:00.864723 614 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260809 07:33:00.896250 31169 raft_consensus.cc:1226] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
W20260809 07:33:00.898144 737 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.133s user 0.000s sys 0.000s
I20260809 07:33:00.813403 632 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260809 07:33:00.898855 32661 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.883288 712 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.874585 30899 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:00.866396 31181 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:00.916811 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.877802 388 tablet_copy_client.cc:655] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Starting download of 1 WAL segments...
I20260809 07:33:00.853587 370 consensus_queue.cc:1046] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260809 07:33:00.925288 338 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:00.937558 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:00.937870 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:00.946946 31006 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:00.930846 31046 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:00.948163 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:01.050050 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 103212us
I20260809 07:33:00.951889 31048 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:00.957628 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:00.961615 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:00.962630 31172 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:01.053562 31172 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:00.935071 31170 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:01.036873 31165 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:00.944593 357 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:00.971467 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:01.040835 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 78017.9us
I20260809 07:33:01.072748 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.046540 31181 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:01.058495 597 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:01.060961 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:01.062108 426 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:00.957238 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.037226 32609 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.062814 318 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:00.969904 510 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:01.067382 32617 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:01.081192 30909 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:01.128239 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 55580.4us
I20260809 07:33:01.082576 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:01.129122 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46641.8us
I20260809 07:33:01.129143 30853 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.956722 713 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.114888 31172 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:01.131608 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:00.951081 30896 raft_consensus.cc:2362] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3. Re-sending same reply.
I20260809 07:33:01.043501 614 consensus_queue.cc:1225] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer 2c9de56c3fa3487c9dac990675c34be5 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:01.119529 31177 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:01.119647 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.116180 646 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.135488 31181 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:01.120879 30852 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:01.122378 748 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.056s user 0.000s sys 0.001s
W20260809 07:33:01.082489 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 120950us
I20260809 07:33:01.067772 30890 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:01.115949 632 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.115828 712 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:01.122479 426 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.067991 630 consensus_queue.cc:1046] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260809 07:33:01.114156 357 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:00.970849 619 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
I20260809 07:33:01.136389 30853 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.160642 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:01.199434 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 357490us
I20260809 07:33:01.202860 31174 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:01.135819 30904 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:01.173965 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.136528 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.249330 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.133767 640 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:33:01.250109 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 113668us
I20260809 07:33:01.271874 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:01.136237 535 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:01.154348 32703 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:01.160840 482 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:01.275930 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.155318 634 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.308159 32661 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:01.309100 602 raft_consensus.cc:3032] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:33:01.309553 602 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.310487 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.127354 603 consensus_queue.cc:1046] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.147s
I20260809 07:33:01.311815 603 raft_consensus.cc:3032] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:33:01.312141 603 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.312949 603 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.314073 603 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:01.222164 426 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.248833 632 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.284263 640 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.288863 31120 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:01.289485 370 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:01.290527 616 consensus_queue.cc:1046] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:01.291491 698 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:01.174930 699 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.118s
I20260809 07:33:01.272523 646 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260809 07:33:01.294732 735 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.074s user 0.000s sys 0.000s
I20260809 07:33:01.296736 357 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:01.302804 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.302187 644 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:01.305507 746 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.266s user 0.002s sys 0.008s
W20260809 07:33:01.306977 752 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.152s user 0.000s sys 0.000s
I20260809 07:33:01.318517 32703 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:01.203013 32609 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.324844 482 raft_consensus.cc:686] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.326800 32717 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:01.326076 602 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.281847 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:01.278355 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 142009us
I20260809 07:33:01.448451 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.449600 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:01.340777 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 51999us
I20260809 07:33:01.451016 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:01.276698 444 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260809 07:33:01.371265 616 raft_consensus.cc:3032] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:01.355628 32596 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:01.352900 698 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.362428 32617 raft_consensus.cc:686] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:01.416087 370 raft_consensus.cc:686] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:01.468232 31048 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:01.454092 30853 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.469686 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.369345 710 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260809 07:33:01.420814 30890 raft_consensus.cc:2362] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3. Re-sending same reply.
I20260809 07:33:01.445969 32717 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:01.473636 30900 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:01.474620 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:01.475147 31058 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:01.477475 31058 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:33:01.478641 31054 raft_consensus.cc:1261] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:01.455343 634 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.479374 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.481007 31051 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260809 07:33:01.482226 31051 pending_rounds.cc:76] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 1
I20260809 07:33:01.482550 31051 pending_rounds.cc:97] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260809 07:33:01.482789 31051 raft_consensus.cc:2866] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:01.455219 616 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:01.461352 650 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 07:33:01.493299 31005 leader_election.cc:334] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 2.166s (SENT)
I20260809 07:33:01.494786 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.474270 441 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.425817 644 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.497582 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.498700 30853 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:01.499034 778 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.472527 32596 raft_consensus.cc:686] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:01.479553 32717 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:01.515859 32596 consensus_queue.cc:237] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.485015 357 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.493986 698 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.505234 31055 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:01.508337 338 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.510171 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 809) took 1443 ms. Trace:
I20260809 07:33:01.471433 768 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:01.486862 602 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:01.480780 664 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 07:33:01.489674 712 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260809 07:33:01.483047 700 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260809 07:33:01.558929 441 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 07:33:01.567168 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.568447 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.572629 644 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:01.574522 569 consensus_queue.cc:1046] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:01.575431 569 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:01.576486 535 raft_consensus.cc:3032] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:01.577163 535 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.577559 32733 raft_consensus.cc:3032] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:01.577947 32733 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.578372 535 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.578724 32733 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.579820 535 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:01.580943 30891 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:01.569610 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.583060 778 raft_consensus.cc:3032] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:33:01.583603 778 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:01.589304 30900 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:01.524915 377 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:01.592334 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.589956 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.594241 30900 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:01.556007 650 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.598498 370 consensus_queue.cc:237] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:01.600253 758 log.cc:924] Time spent T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.170s user 0.000s sys 0.001s
I20260809 07:33:01.600088 444 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:01.530936 30899 rpcz_store.cc:276] 0809 07:33:00.066458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:00.301417 (+234959us) service_pool.cc:227] Handling call
0809 07:33:00.301862 (+ 445us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:01.040116 (+738254us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:01.509894 (+469778us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540045f1d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:01.510107 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1316604416}
I20260809 07:33:01.605033 698 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:01.604821 757 consensus_queue.cc:1046] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:01.607955 30906 raft_consensus.cc:2396] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:01.531011 699 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.488951 634 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.588032 30896 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:01.561184 789 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260809 07:33:01.505852 616 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.618258 31005 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:01.631628 30900 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:01.496248 32703 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.647320 31055 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:01.532416 357 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.651695 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:01.593534 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.607880 768 raft_consensus.cc:686] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:01.620599 387 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.621294 426 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:01.561013 570 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260809 07:33:01.584587 778 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.591387 32661 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260809 07:33:01.662876 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.513586 482 consensus_queue.cc:237] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.623545 351 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260809 07:33:01.554766 774 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.094s
I20260809 07:33:01.631065 664 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.040s
I20260809 07:33:01.656657 644 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
W20260809 07:33:01.669273 318 consensus_meta.cc:287] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.544s user 0.008s sys 0.002s
I20260809 07:33:01.669827 510 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:01.689110 31005 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:01.689165 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53380.5us
I20260809 07:33:01.674403 32596 raft_consensus.cc:3032] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:01.620368 32717 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:01.668601 614 consensus_queue.cc:1159] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a720817fc113446a8ed55e61523b280a" responder_term: 3 status { last_received { term: 3 index: 2 } last_committed_idx: 1 last_received_current_leader { term: 3 index: 2 } }
I20260809 07:33:01.642390 699 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.621090 32617 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.677781 616 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:01.690927 30891 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:01.695072 31174 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:01.693358 387 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:01.668100 640 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260809 07:33:01.694433 377 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.711103 31177 raft_consensus.cc:1261] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:01.712837 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.713587 30900 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:01.694617 32596 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:01.693024 510 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:01.707649 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.727842 31005 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.729254 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.712147 768 consensus_queue.cc:237] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.719787 778 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:01.720642 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.738185 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.724701 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.741823 30900 raft_consensus.cc:2362] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2. Re-sending same reply.
I20260809 07:33:01.702589 644 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:01.740713 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:01.736390 31005 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:01.714277 318 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 07:33:01.731153 792 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.146s user 0.000s sys 0.001s
I20260809 07:33:01.743628 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:01.750123 30900 raft_consensus.cc:2396] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:01.730253 510 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:01.730404 570 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.754571 570 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.756736 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:01.757283 570 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:01.757694 800 log.cc:924] Time spent T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260809 07:33:01.758692 30891 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:01.758939 30890 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:01.748474 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.735237 675 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:01.739286 30896 raft_consensus.cc:1226] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:33:01.763242 699 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:01.722473 426 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260809 07:33:01.763931 30896 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:01.742910 605 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:01.751370 710 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:01.722951 377 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.766685 710 raft_consensus.cc:686] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:01.753333 30911 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:33:01.735531 646 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:33:01.709535 441 consensus_queue.cc:1046] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260809 07:33:01.761500 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.761755 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:01.759598 644 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.770589 30900 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:33:01.730823 700 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260809 07:33:01.790977 31049 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:01.792336 30911 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:01.758754 357 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:01.813563 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52151.3us
I20260809 07:33:01.765455 32717 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:01.765661 318 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:01.814935 31005 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:01.815629 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:01.818274 797 log.cc:924] Time spent T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.121s user 0.000s sys 0.001s
I20260809 07:33:01.794148 640 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:01.819152 651 raft_consensus.cc:3032] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:01.819815 651 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:01.820910 651 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.768555 377 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:01.760231 30890 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
W20260809 07:33:01.819041 809 log.cc:924] Time spent T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.068s user 0.000s sys 0.001s
I20260809 07:33:01.794855 30852 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:01.835574 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.839921 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.765481 30896 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:01.767763 710 consensus_queue.cc:237] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.848393 31005 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.758757 634 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:01.858546 31005 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:01.861186 30853 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:01.877486 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 82737.8us
I20260809 07:33:01.832753 651 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:01.765771 32596 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.880342 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:01.880833 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 617) took 1008 ms. Trace:
I20260809 07:33:01.845501 632 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:01.849454 675 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:01.882397 32721 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 07:33:01.885291 799 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260809 07:33:01.845206 699 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:01.825798 646 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260809 07:33:01.882160 776 raft_consensus.cc:3032] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:01.849671 718 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:01.825570 605 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.909178 32703 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:01.924162 31174 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:01.877372 482 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:01.952103 482 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:01.952986 811 log.cc:924] Time spent T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260809 07:33:01.890807 712 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:01.926812 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:01.828789 640 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.954559 712 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:01.955229 712 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:01.956296 640 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:01.940744 823 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.957073 823 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:01.959666 823 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:01.961968 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 59948.4us
I20260809 07:33:01.941399 510 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:01.943754 603 consensus_meta.cc:287] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.602s user 0.003s sys 0.007s
I20260809 07:33:01.944238 718 raft_consensus.cc:686] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:01.912086 634 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:01.922499 31169 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:01.989434 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:01.918571 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 223683us
I20260809 07:33:01.941278 700 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.917804 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 634) took 1040 ms. Trace:
I20260809 07:33:01.968166 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:01.949465 387 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:01.975823 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:01.937739 32617 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:01.977463 30909 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:01.931850 776 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:02.002097 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 84980.8us
I20260809 07:33:01.970793 823 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:01.946537 32721 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260809 07:33:02.012195 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:01.915624 30890 rpcz_store.cc:276] 0809 07:33:00.871116 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.759671 (+888555us) service_pool.cc:227] Handling call
0809 07:33:01.760045 (+ 374us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:01.845388 (+ 85343us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:01.879086 (+ 33698us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.007048 30852 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:02.033279 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:01.967372 32703 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.007938 591 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:02.010591 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:01.998127 774 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260809 07:33:02.054107 510 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.036805 387 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
W20260809 07:33:02.057122 31175 consensus_meta.cc:287] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.532s user 0.005s sys 0.005s
I20260809 07:33:02.080232 31175 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:02.046968 776 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.032562 823 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.084221 31051 raft_consensus.cc:1261] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:02.084905 823 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.085459 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.032342 675 consensus_queue.cc:1046] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260809 07:33:02.087286 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.087914 591 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.088387 31045 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:02.091089 31045 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:02.055410 482 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.091912 30852 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:02.092222 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 632) took 1215 ms. Trace:
I20260809 07:33:02.003841 30896 rpcz_store.cc:276] 0809 07:33:00.877168 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.764832 (+887664us) service_pool.cc:227] Handling call
0809 07:33:01.765278 (+ 446us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:01.856677 (+ 91399us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:01.909564 (+ 52887us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.092578 30909 rpcz_store.cc:276] 0809 07:33:00.876717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.449270 (+572553us) service_pool.cc:227] Handling call
0809 07:33:02.092074 (+642804us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54003c37d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:02.092169 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1166285568}
I20260809 07:33:02.093878 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.011584 603 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.043397 640 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:02.169607 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:02.095350 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.094887 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:02.095340 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:02.070081 784 log.cc:924] Time spent T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.564s user 0.003s sys 0.000s
W20260809 07:33:02.045758 824 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.139s user 0.003s sys 0.000s
I20260809 07:33:02.177578 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:02.030439 535 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.077121 338 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:02.190701 444 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:02.191272 444 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:02.191880 444 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:02.191931 640 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.192658 640 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.056340 388 tablet_copy_client.cc:531] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:33:02.045957 31169 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:02.201104 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.201942 32733 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:02.092973 482 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:02.054615 778 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.008692 700 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.170691 602 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.185727 825 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.187026 32717 raft_consensus.cc:2741] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260809 07:33:02.168462 634 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:02.182152 31046 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:02.213902 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.214671 31048 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:02.177567 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.215723 30852 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:02.171720 31051 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
W20260809 07:33:02.184478 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 89679.7us
I20260809 07:33:02.178968 619 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.219909 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.184064 30909 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:02.199283 569 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.172392 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.196986 32721 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.239193 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:02.246559 30904 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:02.247160 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 612) took 1377 ms. Trace:
I20260809 07:33:02.248273 30904 rpcz_store.cc:276] 0809 07:33:00.869722 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.238920 (+1369198us) service_pool.cc:227] Handling call
0809 07:33:02.247100 (+ 8180us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.249666 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:02.250581 31005 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:02.254658 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:02.255343 31046 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:02.207160 675 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.043756 718 consensus_queue.cc:237] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.213339 634 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:02.181272 603 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.214025 712 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.236330 591 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.229801 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.292181 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.337205 30911 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:02.338146 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:02.231904 597 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:02.202874 778 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
W20260809 07:33:02.335630 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 76528.1us
W20260809 07:33:02.338362 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 87876.1us
I20260809 07:33:02.339535 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.232573 357 consensus_queue.cc:1046] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260809 07:33:02.259684 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:02.356015 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 101451us
I20260809 07:33:02.448248 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:02.212767 602 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:02.402189 616 consensus_meta.cc:287] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.698s user 0.010s sys 0.008s
I20260809 07:33:02.257159 32721 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.331785 634 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:02.462673 31165 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:02.234448 619 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.444583 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:02.331282 773 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:33:02.256752 30896 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
W20260809 07:33:02.446038 31006 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.67:44107 timed out after 1.572s (SENT) [suppressed 28 similar messages]
I20260809 07:33:02.236029 569 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:02.475474 30906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 602) took 1609 ms (client timeout 1912 ms). Trace:
I20260809 07:33:02.423867 30909 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:02.203805 535 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.336149 675 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.449664 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:02.392961 482 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.400619 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.340466 591 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.470000 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:02.585916 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 116001us
I20260809 07:33:02.446659 441 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.442152 771 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.390038 597 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:02.475898 30906 rpcz_store.cc:269] 0809 07:33:00.866425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.139493 (+273068us) service_pool.cc:227] Handling call
0809 07:33:02.475290 (+1335797us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54001b3ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:02.475423 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":879889664}
I20260809 07:33:02.476881 30909 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:02.478039 31174 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:02.215107 825 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:02.491499 30911 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:02.601919 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 839) took 1136 ms. Trace:
I20260809 07:33:02.469547 387 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:02.477346 616 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:02.537006 651 consensus_meta.cc:287] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.626s user 0.004s sys 0.009s
I20260809 07:33:02.606215 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 838) took 1141 ms. Trace:
I20260809 07:33:02.605223 30904 rpcz_store.cc:276] 0809 07:33:01.465214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.601082 (+1135868us) service_pool.cc:227] Handling call
0809 07:33:02.601430 (+ 348us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.601599 (+ 169us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:02.601623 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:02.601662 (+ 39us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:02.601688 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.601733 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:02.601842 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.631646 30909 rpcz_store.cc:276] 0809 07:33:01.464973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.476426 (+1011453us) service_pool.cc:227] Handling call
0809 07:33:02.476711 (+ 285us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.605957 (+129246us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.606154 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.536726 640 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:02.627523 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 168115us
I20260809 07:33:02.537392 773 raft_consensus.cc:686] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:02.635854 30906 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:02.638617 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 840) took 1173 ms. Trace:
I20260809 07:33:02.638962 30906 rpcz_store.cc:276] 0809 07:33:01.465453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.601696 (+1136243us) service_pool.cc:227] Handling call
0809 07:33:02.635689 (+ 33993us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.638344 (+ 2655us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.638561 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.621554 32566 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.607049 597 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:02.600600 32703 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:02.626359 831 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.374s user 0.001s sys 0.001s
I20260809 07:33:02.642174 31172 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:02.554478 791 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.159s
I20260809 07:33:02.634546 616 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:02.640872 828 log.cc:924] Time spent T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.223s user 0.000s sys 0.002s
I20260809 07:33:02.662125 30906 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:33:02.662781 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 844) took 1195 ms. Trace:
I20260809 07:33:02.663300 619 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.643800 32566 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.667112 31169 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:02.667932 664 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.669219 602 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.640568 650 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:02.595842 482 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.605602 441 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.672639 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 834) took 1208 ms. Trace:
I20260809 07:33:02.663070 30906 rpcz_store.cc:276] 0809 07:33:01.467121 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.639714 (+1172593us) service_pool.cc:227] Handling call
0809 07:33:02.651132 (+ 11418us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.662544 (+ 11412us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.662729 (+ 185us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.599733 591 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:02.675642 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 229640us
I20260809 07:33:02.668699 664 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.661973 616 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:02.680641 318 consensus_meta.cc:287] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.856s user 0.012s sys 0.001s
I20260809 07:33:02.670111 839 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.684082 30900 rpcz_store.cc:276] 0809 07:33:01.463986 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.794613 (+330627us) service_pool.cc:227] Handling call
0809 07:33:01.794924 (+ 311us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:01.827003 (+ 32079us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:01.827038 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:01.832051 (+ 5013us) log.cc:842] Serialized 52 byte log entry
0809 07:33:02.168082 (+336031us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:02.168118 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.168180 (+ 62us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:02.672434 (+504254us) raft_consensus.cc:1671] finished
0809 07:33:02.672455 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:02.672585 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":36359,"raft.queue_time_us":32522,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":27837,"spinlock_wait_cycles":1814784,"thread_start_us":254133,"threads_started":1,"wal-append.queue_time_us":473938}
W20260809 07:33:02.687176 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 237554us
I20260809 07:33:02.698415 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:02.606801 771 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.660063 712 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.695876 30906 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:02.705780 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 639) took 1024 ms. Trace:
I20260809 07:33:02.654680 700 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.674980 30890 raft_consensus.cc:1261] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:02.701953 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.672277 32566 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.706121 30906 rpcz_store.cc:276] 0809 07:33:01.681275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.684325 (+1003050us) service_pool.cc:227] Handling call
0809 07:33:02.695542 (+ 11217us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.705527 (+ 9985us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.705725 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.678804 32721 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.713380 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.725396 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.695338 646 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:02.693503 603 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.736477 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:02.737114 31174 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:02.690430 650 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:02.698577 318 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:02.750849 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.752744 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 607) took 1884 ms. Trace:
I20260809 07:33:02.753374 30899 rpcz_store.cc:276] 0809 07:33:00.868258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.753685 (+885427us) service_pool.cc:227] Handling call
0809 07:33:01.849377 (+ 95692us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:01.948580 (+ 99203us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:01.962120 (+ 13540us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:01.962739 (+ 619us) log.cc:842] Serialized 26 byte log entry
0809 07:33:02.024895 (+ 62156us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:02.035070 (+ 10175us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.099228 (+ 64158us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:02.730726 (+631498us) raft_consensus.cc:1671] finished
0809 07:33:02.733387 (+ 2661us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:02.752660 (+ 19273us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":45227,"raft.queue_time_us":52958,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":172,"thread_start_us":43366,"threads_started":1,"wal-append.queue_time_us":218662}
I20260809 07:33:02.756954 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.759203 825 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:02.759964 825 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.760733 825 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.720350 700 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:02.762835 700 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.763896 482 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.765385 700 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:02.705700 31175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 674) took 1951 ms (client timeout 1572 ms). Trace:
I20260809 07:33:02.766026 842 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:02.767277 577 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.767957 577 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.743238 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.744406 858 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:02.745381 591 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.747211 441 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:02.749740 832 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.287s user 0.000s sys 0.001s
I20260809 07:33:02.716845 357 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:02.739581 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:02.721016 834 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.279s user 0.001s sys 0.001s
I20260809 07:33:02.720386 32566 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.723843 849 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.806594 31051 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:02.801357 31174 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:02.719153 778 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.813405 829 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:02.792908 338 consensus_meta.cc:287] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.605s user 0.010s sys 0.002s
I20260809 07:33:02.780226 619 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:02.770630 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 837) took 1305 ms. Trace:
I20260809 07:33:02.827418 31048 raft_consensus.cc:1261] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:02.787829 791 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:02.832541 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 136045us
W20260809 07:33:02.829731 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 100029us
I20260809 07:33:02.831668 31164 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:02.787568 535 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.835772 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:02.838615 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.795090 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:02.810158 32733 consensus_meta.cc:287] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.608s user 0.008s sys 0.004s
I20260809 07:33:02.839277 31174 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:02.802997 357 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:02.804937 675 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.808650 569 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.842161 30853 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:02.842677 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.741217 646 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.854919 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.710971 773 consensus_queue.cc:237] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.851991 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.770206 650 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.850142 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.707159 839 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:02.856215 31174 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
W20260809 07:33:02.766047 31175 rpcz_store.cc:269] 0809 07:33:00.748351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:00.909656 (+161305us) service_pool.cc:227] Handling call
0809 07:33:02.693637 (+1783981us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54003d0c54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:02.699904 (+ 6267us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1381601152}
I20260809 07:33:02.866902 31054 raft_consensus.cc:2396] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:02.863945 30853 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:02.845286 602 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:02.868942 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.843065 619 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:02.820478 849 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.844964 338 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:02.883325 444 consensus_meta.cc:287] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.691s user 0.011s sys 0.002s
I20260809 07:33:02.883147 338 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.883860 444 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.886734 602 raft_consensus.cc:686] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:02.875478 30906 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:02.870258 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.822126 32566 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:02.864738 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 166407us
I20260809 07:33:02.814430 829 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.850595 568 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.847158 32703 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:02.875559 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:02.873282 858 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:02.876797 646 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.861428 778 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:02.910827 858 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:02.832836 30890 rpcz_store.cc:276] 0809 07:33:01.464736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.471676 (+1006940us) service_pool.cc:227] Handling call
0809 07:33:02.641709 (+170033us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.711444 (+ 69735us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.742175 (+ 30731us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:02.874554 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.888336 602 consensus_queue.cc:237] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.888757 634 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.894651 357 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.897473 31169 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:02.876714 535 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.904901 650 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.902442 838 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.842082 791 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.914825 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.913405 31165 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:02.890205 825 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:02.912372 778 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:02.908191 338 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:02.912479 858 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:02.903286 482 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.921916 569 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.922263 444 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:02.913080 568 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.907549 32733 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.907262 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:02.909229 664 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.026600 569 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.038985 664 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:03.039422 30898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 599) took 2174 ms (client timeout 2034 ms). Trace:
I20260809 07:33:03.039660 664 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.913607 577 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:02.938405 619 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.938480 644 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:02.935738 849 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:02.983726 838 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.983451 31046 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:02.982059 825 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:02.949656 535 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.948127 771 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.054118 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.055436 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.056128 31046 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.057009 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:02.930259 388 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Bootstrap starting.
I20260809 07:33:02.983106 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.000466 868 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.003976 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:02.915612 30899 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:02.929441 30852 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:03.063788 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 134510us
I20260809 07:33:03.064810 30852 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:02.939147 591 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:03.019134 861 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.149s user 0.001s sys 0.000s
W20260809 07:33:03.020478 810 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.061s user 0.000s sys 0.007s
I20260809 07:33:02.997402 778 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.024127 32566 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:02.912992 32703 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:02.908252 510 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260809 07:33:03.007457 357 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:03.033886 318 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.034595 32733 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.057648 31046 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:03.017529 482 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.071439 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 820) took 1599 ms. Trace:
I20260809 07:33:03.088850 31059 rpcz_store.cc:276] 0809 07:33:01.467839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.468586 (+ 747us) service_pool.cc:227] Handling call
0809 07:33:01.468937 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:01.553451 (+ 84514us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:01.553486 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:01.553532 (+ 46us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:01.553560 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.067156 (+1513596us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b540031b854. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:03.067204 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.067333 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":193869184}
I20260809 07:33:03.090773 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.092058 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.094035 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.094786 31046 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:03.095534 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.096050 31046 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.072522 441 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:33:03.073962 646 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.042219 616 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.056738 644 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.097445 876 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.093364 318 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.096748 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.102118 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.082911 32566 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:03.078218 30890 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:03.119257 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.128818 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 842) took 1662 ms. Trace:
I20260809 07:33:03.132380 31046 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:33:03.062785 825 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.135808 31059 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
W20260809 07:33:03.053416 30898 rpcz_store.cc:269] 0809 07:33:00.865470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:00.867214 (+ 1744us) service_pool.cc:227] Handling call
0809 07:33:03.039176 (+2171962us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54003b9054. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:03.039356 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1229154176}
I20260809 07:33:03.136207 30904 rpcz_store.cc:276] 0809 07:33:01.465924 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.632449 (+1166525us) service_pool.cc:227] Handling call
0809 07:33:02.632756 (+ 307us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.632968 (+ 212us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:33:02.824353 (+191385us) log.cc:842] Serialized 14 byte log entry
0809 07:33:03.128490 (+304137us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.128540 (+ 50us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:03.128579 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.128643 (+ 64us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.128762 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2708,"prepare.queue_time_us":364798,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":5991,"raft.queue_time_us":408,"thread_start_us":474757,"threads_started":2}
I20260809 07:33:03.144636 31174 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:03.073027 591 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.145301 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.108295 664 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.104072 32733 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.150624 31005 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:03.153870 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 640) took 1472 ms. Trace:
I20260809 07:33:03.155846 30900 rpcz_store.cc:276] 0809 07:33:01.681553 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.697951 (+1016398us) service_pool.cc:227] Handling call
0809 07:33:02.698276 (+ 325us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.698561 (+ 285us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:02.819144 (+120583us) log.cc:842] Serialized 14 byte log entry
0809 07:33:03.153535 (+334391us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.153591 (+ 56us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:03.153632 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.153692 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.153813 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":329285,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":953,"thread_start_us":432922,"threads_started":2}
I20260809 07:33:03.161079 829 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.164496 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.164889 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 642) took 1481 ms. Trace:
I20260809 07:33:03.166345 30900 rpcz_store.cc:276] 0809 07:33:01.683367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.163902 (+1480535us) service_pool.cc:227] Handling call
0809 07:33:03.164313 (+ 411us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:03.164489 (+ 176us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:03.164509 (+ 20us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.164612 (+ 103us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:03.164649 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.164709 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.164832 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:03.139631 32717 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:03.149480 839 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.172524 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 845) took 1174 ms. Trace:
I20260809 07:33:03.117116 646 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.176090 838 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.152266 806 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.177664 30900 rpcz_store.cc:276] 0809 07:33:01.998560 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.169826 (+1171266us) service_pool.cc:227] Handling call
0809 07:33:03.170111 (+ 285us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:03.170278 (+ 167us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:03.171776 (+ 1498us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.171829 (+ 53us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:03.171865 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.172360 (+ 495us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.172469 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:03.154244 791 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.180639 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 751) took 1493 ms. Trace:
I20260809 07:33:03.170095 510 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.154547 634 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.169040 855 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.189522 30900 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:03.191669 870 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.185273 31182 rpcz_store.cc:276] 0809 07:33:01.687302 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.687450 (+ 148us) service_pool.cc:227] Handling call
0809 07:33:01.687761 (+ 311us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:01.688089 (+ 328us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:01.688115 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:01.688511 (+ 396us) log.cc:842] Serialized 26 byte log entry
0809 07:33:01.741942 (+ 53431us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:01.750878 (+ 8936us) log.cc:842] Serialized 14 byte log entry
0809 07:33:01.791616 (+ 40738us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:01.791683 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:02.925278 (+1133595us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:03.173388 (+248110us) raft_consensus.cc:1671] finished
0809 07:33:03.180434 (+ 7046us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.180575 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":81057,"prepare.queue_time_us":10689,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":63244,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":49609,"spinlock_wait_cycles":22810880,"thread_start_us":40614,"threads_started":1,"wal-append.queue_time_us":129725}
I20260809 07:33:03.200585 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.212697 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 855) took 1212 ms. Trace:
I20260809 07:33:03.187987 510 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.231180 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.231451 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.237761 31059 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:03.238056 31046 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:03.166463 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 643) took 1482 ms. Trace:
I20260809 07:33:03.231967 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:03.207340 30852 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:03.248564 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:03.249281 30852 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:03.233918 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 846) took 1230 ms. Trace:
I20260809 07:33:03.250483 30900 rpcz_store.cc:276] 0809 07:33:02.003237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.188927 (+1185690us) service_pool.cc:227] Handling call
0809 07:33:03.189292 (+ 365us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:03.214151 (+ 24859us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.233860 (+ 19709us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:03.251703 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.198477 568 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.233035 31058 rpcz_store.cc:276] 0809 07:33:02.000464 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.000731 (+ 267us) service_pool.cc:227] Handling call
0809 07:33:03.200439 (+1199708us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b5800004598. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:33:03.212629 (+ 12190us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7709440}
I20260809 07:33:03.246970 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.262204 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.178465 32717 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.254469 30852 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:03.230459 569 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.083282 868 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.228564 634 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:03.262954 31182 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.295504 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.295845 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.296439 31182 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.297271 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.262605 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.267436 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:03.239547 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.129460 876 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.233911 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 837) took 1225 ms. Trace:
I20260809 07:33:03.246654 510 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.347061 318 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.352233 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.298887 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.354048 591 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.354627 30900 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:03.344318 577 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260809 07:33:03.355522 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.356145 30900 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:03.299232 31059 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.328881 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:03.356674 885 log.cc:924] Time spent T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.123s user 0.000s sys 0.003s
I20260809 07:33:03.356379 577 consensus_queue.cc:1046] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:03.357103 569 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.357628 577 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.330678 634 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.254686 441 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.252652 535 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.271548 839 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:03.368937 31058 raft_consensus.cc:2396] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:03.332590 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.370230 31046 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.371170 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.371767 31046 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:33:03.372532 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:03.267320 377 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.777s user 0.003s sys 0.032s
I20260809 07:33:03.253062 32733 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.449415 32733 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:03.349534 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 54367.3us
I20260809 07:33:03.452085 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:03.452941 30853 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:03.453061 32733 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.361107 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.454603 31059 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:03.454989 839 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.262590 32703 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.454562 535 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:03.364157 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 848) took 1339 ms. Trace:
I20260809 07:33:03.456041 535 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.455519 839 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.456085 30900 rpcz_store.cc:276] 0809 07:33:02.024279 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.356737 (+1332458us) service_pool.cc:227] Handling call
0809 07:33:03.363764 (+ 7027us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:03.363900 (+ 136us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:03.363921 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.363956 (+ 35us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:03.363982 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.364022 (+ 40us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.364114 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:03.457165 839 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.457595 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.359900 802 raft_consensus.cc:3032] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:03.366583 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.459255 31165 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:03.455600 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.444598 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:03.354647 891 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.057s user 0.001s sys 0.000s
I20260809 07:33:03.270653 664 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:03.269631 838 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:33:03.461462 31046 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 1.
W20260809 07:33:03.375774 597 consensus_meta.cc:287] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.717s user 0.008s sys 0.005s
I20260809 07:33:03.473420 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.464891 890 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.233186 855 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.468413 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.357651 569 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:03.462688 377 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.973s user 0.011s sys 0.034s
I20260809 07:33:03.474519 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.367988 441 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.442478 357 raft_consensus.cc:3032] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:03.362296 616 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.299880 568 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.148s
I20260809 07:33:03.247786 30890 rpcz_store.cc:276] 0809 07:33:01.683621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.165516 (+1481895us) service_pool.cc:227] Handling call
0809 07:33:03.165855 (+ 339us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:03.166028 (+ 173us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:03.166054 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.166096 (+ 42us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:03.166126 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.166183 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.166371 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:03.458370 802 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:03.506978 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.296572 833 consensus_queue.cc:1046] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.173s
I20260809 07:33:03.471962 838 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.528065 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.528705 31059 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
W20260809 07:33:03.505158 877 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.118s user 0.001s sys 0.000s
I20260809 07:33:03.463490 535 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:03.532043 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 81829.5us
I20260809 07:33:03.362036 577 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.506891 569 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.506326 441 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.522800 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.523553 880 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.524755 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:03.229948 870 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.535636 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:03.525560 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 271145us
I20260809 07:33:03.529664 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.532305 30904 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:03.539367 31165 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:03.506631 32703 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.537019 31182 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:03.478032 825 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.539989 31059 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:03.540755 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:03.541666 30853 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:03.509966 357 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:03.552714 31059 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:03.531620 634 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.557063 30853 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:03.477936 597 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.531162 838 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:03.563637 602 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.564229 602 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.349289 31045 rpcz_store.cc:276] 0809 07:33:02.008586 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.092190 (+ 83604us) service_pool.cc:227] Handling call
0809 07:33:02.176189 (+ 83999us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:02.214026 (+ 37837us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:02.229384 (+ 15358us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:02.476403 (+247019us) log.cc:842] Serialized 52 byte log entry
0809 07:33:02.708577 (+232174us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:02.792224 (+ 83647us) log.cc:842] Serialized 14 byte log entry
0809 07:33:02.792551 (+ 327us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:02.792631 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:03.192202 (+399571us) raft_consensus.cc:1671] finished
0809 07:33:03.193962 (+ 1760us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.230187 (+ 36225us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13,"prepare.queue_time_us":67657,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":7060,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":36861,"spinlock_wait_cycles":28491520,"thread_start_us":59606,"threads_started":1,"wal-append.queue_time_us":223248}
I20260809 07:33:03.507984 858 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.564761 602 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.552029 646 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.551657 712 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:03.574216 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:03.577644 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:03.520453 890 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:03.571517 878 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.183s user 0.000s sys 0.001s
I20260809 07:33:03.556720 876 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.586555 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:03.588641 778 consensus_meta.cc:287] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.505s user 0.006s sys 0.006s
I20260809 07:33:03.588965 616 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:03.574498 712 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.556906 32703 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.582589 32717 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.591032 568 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:03.589174 650 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:03.607584 616 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.607731 591 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:03.621529 892 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
W20260809 07:33:03.612421 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 71728us
I20260809 07:33:03.581380 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:03.627539 577 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.620404 898 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:03.631686 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 836) took 2167 ms. Trace:
I20260809 07:33:03.587057 825 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260809 07:33:03.636996 883 log.cc:924] Time spent T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.269s user 0.001s sys 0.000s
I20260809 07:33:03.630687 791 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:03.603593 655 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.640395 616 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:03.649667 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:03.671470 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:03.650604 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.632210 712 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:03.633607 568 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.639915 591 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.603703 868 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.648818 30911 rpcz_store.cc:276] 0809 07:33:01.464495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.472581 (+1008086us) service_pool.cc:227] Handling call
0809 07:33:02.491277 (+ 18696us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:02.605070 (+113793us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.631492 (+1026422us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b5400281250. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:03.631628 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17907072}
I20260809 07:33:03.687517 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.633875 650 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.619946 855 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.668195 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:03.677817 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 201988us
I20260809 07:33:03.571283 880 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.661141 30890 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.608187 778 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:03.665221 791 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.595808 357 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.703634 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.659917 870 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.709228 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:03.714113 627 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:03.714434 896 log.cc:924] Time spent T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.092s user 0.002s sys 0.000s
I20260809 07:33:03.715855 806 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:03.717583 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 655) took 1005 ms. Trace:
I20260809 07:33:03.719125 858 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.586344 897 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260809 07:33:03.725972 31059 raft_consensus.cc:2396] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:03.686555 568 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:03.729888 893 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.188s user 0.001s sys 0.000s
I20260809 07:33:03.719015 634 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:03.693066 650 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:03.730425 32617 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.775264 32617 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:03.759618 886 log.cc:924] Time spent T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.232s user 0.000s sys 0.002s
I20260809 07:33:03.707088 894 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:03.734584 31045 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:03.751852 597 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.751729 627 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.754022 591 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.722608 855 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:03.720185 32717 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.759375 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 758) took 1790 ms. Trace:
I20260809 07:33:03.726424 791 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:03.803071 31181 rpcz_store.cc:276] 0809 07:33:01.968478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.231564 (+263086us) service_pool.cc:227] Handling call
0809 07:33:02.338928 (+107364us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:02.842493 (+503565us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:02.842521 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:02.847906 (+ 5385us) log.cc:842] Serialized 26 byte log entry
0809 07:33:03.146156 (+298250us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:03.146190 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.146260 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:03.759164 (+612904us) raft_consensus.cc:1671] finished
0809 07:33:03.759186 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.759308 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":105940,"raft.queue_time_us":106517,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":10732,"thread_start_us":674718,"threads_started":2,"wal-append.queue_time_us":380392}
I20260809 07:33:03.694073 890 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.718114 569 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.754071 806 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.763728 357 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:03.804364 32717 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.755414 30899 rpcz_store.cc:276] 0809 07:33:02.712503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.756535 (+ 44032us) service_pool.cc:227] Handling call
0809 07:33:03.717367 (+960832us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54003c28d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:03.717507 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":352475904}
I20260809 07:33:03.777505 778 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.829857 880 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:03.813052 597 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.801049 634 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.834867 32617 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.803949 855 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.837591 627 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:03.837033 821 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 1.613s user 0.018s sys 0.000s
I20260809 07:33:03.678160 655 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.838721 32717 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.813846 894 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.905540 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:03.864657 855 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:03.908636 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:03.906821 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:03.917330 627 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:03.856961 634 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.901219 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 688) took 1971 ms. Trace:
I20260809 07:33:03.912106 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:03.918833 32617 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:03.921008 31181 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:03.887090 894 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:03.922041 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 321937us
I20260809 07:33:03.856796 597 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.917768 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:03.941531 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 850) took 2164 ms. Trace:
I20260809 07:33:03.927074 32617 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:03.929436 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:03.918119 627 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:03.944519 31055 rpcz_store.cc:276] 0809 07:33:01.772652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.772794 (+ 142us) service_pool.cc:227] Handling call
0809 07:33:01.773072 (+ 278us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:01.773393 (+ 321us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:01.773420 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:01.794552 (+ 21132us) log.cc:842] Serialized 26 byte log entry
0809 07:33:01.933685 (+139133us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:01.933726 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:01.933811 (+ 85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:03.829745 (+1895934us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:03.935176 (+105431us) raft_consensus.cc:1671] finished
0809 07:33:03.935201 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.936786 (+ 1585us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":14970,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":21237,"spinlock_wait_cycles":27184000,"thread_start_us":102254,"threads_started":1,"wal-append.queue_time_us":221001}
I20260809 07:33:03.952064 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.952744 31055 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:03.953707 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.954282 31055 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:03.955072 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.932919 903 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:03.964295 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:03.959836 648 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:03.946718 32617 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:03.949517 31169 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:33:03.956329 510 consensus_meta.cc:287] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.601s user 0.013s sys 0.000s
I20260809 07:33:03.945416 32703 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:03.992121 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 320713us
I20260809 07:33:03.992895 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:03.962136 634 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.966235 597 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.014099 838 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:03.995513 710 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.110s user 0.006s sys 0.025s
I20260809 07:33:04.025121 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:04.031745 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 95578.8us
I20260809 07:33:04.000960 32566 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:04.033265 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:03.993947 569 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.919766 31170 rpcz_store.cc:276] 0809 07:33:01.923408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.926464 (+ 3056us) service_pool.cc:227] Handling call
0809 07:33:01.973073 (+ 46609us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:02.025218 (+ 52145us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:02.034495 (+ 9277us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:02.057719 (+ 23224us) log.cc:842] Serialized 26 byte log entry
0809 07:33:02.918500 (+860781us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b1800379d80. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411 00007f4094ecd5cf
0809 07:33:03.248396 (+329896us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:03.260914 (+ 12518us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:03.280995 (+ 20081us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:03.865228 (+584233us) raft_consensus.cc:1671] finished
0809 07:33:03.869899 (+ 4671us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:03.894500 (+ 24601us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":102933,"raft.queue_time_us":18250,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2565,"spinlock_wait_cycles":20293376,"thread_start_us":312749,"threads_started":1,"wal-append.queue_time_us":1385105}
I20260809 07:33:04.007972 602 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.036441 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.038110 30852 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:04.040138 30900 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
W20260809 07:33:04.032503 710 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.144s user 0.006s sys 0.026s
I20260809 07:33:04.025084 32703 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.042546 30852 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:04.047261 31181 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:03.991115 510 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.059556 30852 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:04.049005 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.049708 30911 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:04.057339 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:04.034969 899 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:04.061172 30852 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:04.033447 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:04.062160 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.005393 616 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.043895 32566 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:04.041082 597 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:04.039876 825 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:04.024291 441 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:04.114398 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 50165.9us
I20260809 07:33:04.114890 31046 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:04.049981 655 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:03.994191 903 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.114763 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:04.125085 31170 raft_consensus.cc:2396] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:04.125765 903 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.119488 597 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:04.123593 655 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:04.090715 569 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:04.119673 825 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:04.114575 855 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.126715 31045 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:04.127833 569 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:04.128943 569 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.129455 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.130856 30899 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:04.127877 825 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:04.131875 569 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.126989 597 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:04.132586 31045 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:04.134563 614 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:04.128283 655 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.135664 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.134938 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.060715 634 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:04.079914 899 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.142925 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.063891 823 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:04.080426 897 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:04.143604 31045 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:04.130070 855 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.154610 441 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.155923 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 831) took 2161 ms. Trace:
I20260809 07:33:04.157789 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:04.120136 32566 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:04.160112 602 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.160353 838 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.160571 31049 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:04.042435 928 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260809 07:33:04.172158 31182 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:04.171048 31057 rpcz_store.cc:276] 0809 07:33:01.994436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.994594 (+ 158us) service_pool.cc:227] Handling call
0809 07:33:01.994911 (+ 317us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.176915 (+182004us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:02.176952 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:02.176997 (+ 45us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:02.177027 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:04.155660 (+1978633us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400339fd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:04.155723 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:04.155861 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":418156416}
I20260809 07:33:04.058640 32703 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.175671 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 835) took 2167 ms. Trace:
I20260809 07:33:04.173377 32566 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:04.180497 31049 rpcz_store.cc:276] 0809 07:33:02.007895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.009554 (+ 1659us) service_pool.cc:227] Handling call
0809 07:33:04.160116 (+2150562us) spinlock_profiling.cc:243] Waited 995 us on lock 0x7b5800004598. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:33:04.160325 (+ 209us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:04.175397 (+ 15072us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:04.175616 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2288512}
I20260809 07:33:04.177996 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:04.164439 839 consensus_meta.cc:287] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.707s user 0.005s sys 0.009s
I20260809 07:33:04.186043 31045 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:04.171929 897 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:04.164958 903 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.196165 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.196594 31057 raft_consensus.cc:2396] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:04.193008 919 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:04.197806 510 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.199275 627 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.202474 807 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:04.205667 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:04.211544 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 75957.1us
I20260809 07:33:04.207680 31045 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:04.213131 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:04.220940 30848 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43222 dropped due to backpressure. The service queue is full; it has 50 items. [suppressed 5 similar messages]
I20260809 07:33:04.218305 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.223559 30890 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:04.225054 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.226542 602 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.213868 644 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.227239 602 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.229243 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.209741 890 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:04.230630 32609 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.230917 890 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.231393 32609 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.229907 602 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.218698 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.207777 897 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:04.235411 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.235664 31120 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:04.242233 31045 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:33:04.243129 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.244258 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.244963 31045 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:04.245764 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.148689 712 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.217087 840 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:04.253052 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:04.253934 819 log.cc:924] Time spent T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.390s user 0.004s sys 0.000s
I20260809 07:33:04.255041 338 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:04.256078 904 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:04.230439 823 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.256846 32617 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.257505 823 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.205085 839 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:04.214107 919 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.218254 918 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:04.259639 899 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:04.261152 535 consensus_meta.cc:287] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.722s user 0.009s sys 0.003s
I20260809 07:33:04.261281 655 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.262805 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 783) took 1402 ms. Trace:
I20260809 07:33:04.262987 597 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.264312 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 753) took 2518 ms. Trace:
I20260809 07:33:04.264137 791 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.264933 890 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.253655 31045 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:04.230021 30899 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:04.239910 807 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:04.359238 31054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 834) took 2268 ms (client timeout 1746 ms). Trace:
I20260809 07:33:04.256536 922 raft_consensus.cc:2741] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 07:33:04.241498 644 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:04.214982 903 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.382210 30852 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:04.487864 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 105728us
I20260809 07:33:04.488445 31165 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:04.489595 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.265214 482 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:04.408309 31006 leader_election.cc:334] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Timed out: RequestConsensusVote RPC to 127.28.87.67:44107 timed out after 1.572s (SENT)
I20260809 07:33:04.409781 31174 rpcz_store.cc:276] 0809 07:33:02.859990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.901431 (+ 41441us) service_pool.cc:227] Handling call
0809 07:33:02.901739 (+ 308us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:03.148792 (+247053us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:04.262540 (+1113748us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54000c5b54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:04.262742 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":558120704}
I20260809 07:33:04.251674 602 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.256776 904 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:04.424564 921 log.cc:924] Time spent T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.140s user 0.001s sys 0.000s
I20260809 07:33:04.426790 32617 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.385041 898 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:04.435786 937 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:04.436297 32703 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.437613 32566 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.434638 776 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:04.438629 823 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.358783 840 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.490805 31006 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:04.491878 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.495595 31045 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:04.483278 807 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:04.499781 602 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.500509 712 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.415267 627 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:04.503017 31054 rpcz_store.cc:269] 0809 07:33:01.997080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.997218 (+ 138us) service_pool.cc:227] Handling call
0809 07:33:02.750697 (+753479us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b5800004598. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e3b1b0 00007f4099e3aa94 00007f4093522c04 00007f4093522b92
0809 07:33:04.265395 (+1514698us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400394754. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:04.265522 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":275289216}
I20260809 07:33:04.359998 796 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.414s
W20260809 07:33:04.499660 934 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.259s user 0.001s sys 0.000s
I20260809 07:33:04.505895 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.411906 535 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.567827 32566 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.569589 597 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.570541 602 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.413398 906 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:04.571611 650 consensus_meta.cc:287] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.776s user 0.010s sys 0.003s
I20260809 07:33:04.572685 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.572166 650 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.573894 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.501252 904 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.496723 644 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.413978 31177 rpcz_store.cc:276] 0809 07:33:01.745412 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.745575 (+ 163us) service_pool.cc:227] Handling call
0809 07:33:01.745889 (+ 314us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:01.789999 (+ 44110us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:33:01.821083 (+ 31084us) log.cc:842] Serialized 14 byte log entry
0809 07:33:01.913515 (+ 92432us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:01.914344 (+ 829us) log.cc:842] Serialized 26 byte log entry
0809 07:33:01.914466 (+ 122us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:01.914675 (+ 209us) log.cc:842] Serialized 14 byte log entry
0809 07:33:01.915027 (+ 352us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:01.915096 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:02.200338 (+285242us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b1800029320. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:02.755620 (+555282us) spinlock_profiling.cc:243] Waited 9.23 ms on lock 0x7b1800029320. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:04.129472 (+1373852us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b1800029320. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:04.145431 (+ 15959us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:04.264095 (+118664us) raft_consensus.cc:1671] finished
0809 07:33:04.264117 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:04.264243 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":62437,"prepare.queue_time_us":31745,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":34442,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":35591,"spinlock_wait_cycles":66109312,"thread_start_us":82678,"threads_started":1,"wal-append.queue_time_us":92523}
I20260809 07:33:04.423524 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.515616 804 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.516438 919 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.483306 890 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:04.421036 918 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.428539 616 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:04.462527 31169 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:04.502185 655 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:04.593268 655 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:04.593861 655 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:04.594952 644 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.407437 756 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.111s
I20260809 07:33:04.502066 774 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:04.562760 627 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.566393 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.567847 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.575637 30911 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:04.602447 31055 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:04.602978 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.569437 535 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.543843 32703 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:04.239004 30848 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43222 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 868)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 876)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 672)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 870)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 875)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 879)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 684)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 671)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 668)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 878)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 850)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 851)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 650)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 652)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 855)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 856)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 653)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 659)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 857)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 858)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 872)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 873)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 874)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 877)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 669)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 674)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 880)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 680)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 681)
I20260809 07:33:04.605728 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.606566 31055 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:04.587422 804 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:04.587009 32717 consensus_meta.cc:287] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.694s user 0.010s sys 0.002s
I20260809 07:33:04.543215 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:04.605700 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 113886us
I20260809 07:33:04.605942 31054 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 2: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:04.611526 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.612274 31054 raft_consensus.cc:2396] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:04.610302 30852 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:04.615389 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:04.603739 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 195466us
I20260809 07:33:04.588136 30890 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:04.605571 774 raft_consensus.cc:686] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:04.517390 937 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.571620 906 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.607782 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 866) took 1247 ms. Trace:
I20260809 07:33:04.621747 602 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.628080 650 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.634420 602 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.627795 712 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:04.606648 756 consensus_queue.cc:1225] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260809 07:33:04.505815 898 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.631126 30911 rpcz_store.cc:276] 0809 07:33:03.360207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.602714 (+1242507us) service_pool.cc:227] Handling call
0809 07:33:04.607716 (+ 5002us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:04.622756 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:04.652943 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:04.587572 890 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.616124 597 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:04.653285 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.659699 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.647218 906 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.656734 918 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:04.660667 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 870) took 1216 ms. Trace:
I20260809 07:33:04.663166 30911 rpcz_store.cc:276] 0809 07:33:03.444694 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.652661 (+1207967us) service_pool.cc:227] Handling call
0809 07:33:04.660589 (+ 7928us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:04.664137 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.636929 839 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.635643 32703 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:04.665298 31055 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:04.668946 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.664778 30911 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:04.670493 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 860) took 1312 ms. Trace:
I20260809 07:33:04.670781 30911 rpcz_store.cc:276] 0809 07:33:03.358427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.663858 (+1305431us) service_pool.cc:227] Handling call
0809 07:33:04.670434 (+ 6576us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:04.671871 30852 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:04.672019 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.672528 840 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.617151 32717 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.676172 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.676977 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:04.654870 791 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:04.637431 897 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.634820 774 consensus_queue.cc:237] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:04.763000 30906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 656) took 2050 ms (client timeout 1520 ms). Trace:
W20260809 07:33:05.012651 30906 rpcz_store.cc:269] 0809 07:33:02.712770 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.721285 (+ 8515us) service_pool.cc:227] Handling call
0809 07:33:04.762772 (+2041487us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b54003ad9d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:04.762917 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":165803392}
I20260809 07:33:04.891927 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 863) took 2035 ms. Trace:
I20260809 07:33:05.015074 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:04.657637 627 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:04.892750 30904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 662) took 1794 ms (client timeout 1823 ms). Trace:
W20260809 07:33:04.778800 31049 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.072s sys 0.114s
I20260809 07:33:04.687623 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.015705 31055 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:04.655604 712 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:05.015184 31048 rpcz_store.cc:276] 0809 07:33:02.856497 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.864576 (+ 8079us) service_pool.cc:227] Handling call
0809 07:33:02.890802 (+ 26226us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:03.099182 (+208380us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:03.099217 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:03.099258 (+ 41us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:03.099290 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:04.891625 (+1792335us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400070654. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:04.891683 (+ 58us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:04.891852 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":478732160}
I20260809 07:33:05.017814 651 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.721935 535 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:04.695340 30890 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:04.722216 32566 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:05.019016 807 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.662940 597 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:05.060572 31045 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:05.003268 655 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:04.668123 823 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 07:33:05.061419 30904 rpcz_store.cc:269] 0809 07:33:03.098461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.144288 (+ 45827us) service_pool.cc:227] Handling call
0809 07:33:04.892557 (+1748269us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001a4ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:04.892686 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":879648512}
I20260809 07:33:05.084157 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:04.958331 888 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s user 0.045s sys 0.105s
W20260809 07:33:04.821198 825 consensus_meta.cc:287] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.689s user 0.008s sys 0.005s
I20260809 07:33:04.668318 650 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
W20260809 07:33:05.062810 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 45256.9us
I20260809 07:33:05.080128 535 raft_consensus.cc:686] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:04.774273 806 consensus_meta.cc:287] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.937s user 0.011s sys 0.004s
W20260809 07:33:04.727659 568 consensus_meta.cc:287] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.949s user 0.011s sys 0.001s
I20260809 07:33:04.826766 949 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.782656 675 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:05.072764 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.101785 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 865) took 1741 ms. Trace:
I20260809 07:33:05.102115 30906 rpcz_store.cc:276] 0809 07:33:03.359893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.060856 (+1700963us) service_pool.cc:227] Handling call
0809 07:33:05.101722 (+ 40866us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:04.780738 30891 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:04.993901 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.757787 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:04.772716 890 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:04.688012 936 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:04.701733 839 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:05.062911 906 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:05.076383 31055 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:04.701572 804 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.080206 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.016390 903 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:05.076471 768 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.115s user 0.026s sys 0.024s
I20260809 07:33:04.685191 32703 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.087778 31181 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 5. Ops: []
I20260809 07:33:05.072371 510 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:05.084756 30904 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:05.082341 597 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:05.082013 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 672) took 1132 ms. Trace:
I20260809 07:33:05.093566 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:05.063707 31049 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s user 0.075s sys 0.117s
I20260809 07:33:05.133180 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 871) took 1208 ms. Trace:
W20260809 07:33:05.087241 888 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s user 0.057s sys 0.113s
I20260809 07:33:05.086179 823 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:05.133446 31049 rpcz_store.cc:276] 0809 07:33:03.924296 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.249033 (+324737us) service_pool.cc:227] Handling call
0809 07:33:04.249367 (+ 334us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:04.249678 (+ 311us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:04.251508 (+ 1830us) log.cc:842] Serialized 14 byte log entry
0809 07:33:05.132839 (+881331us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:05.132890 (+ 51us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:05.132928 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.133013 (+ 85us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:05.133123 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":138,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":5542,"thread_start_us":880999,"threads_started":1,"wal-append.queue_time_us":831294}
I20260809 07:33:05.137028 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.137461 31049 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 3
I20260809 07:33:05.079563 32566 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.097777 949 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.099231 807 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:05.100600 776 consensus_meta.cc:287] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.560s user 0.004s sys 0.007s
I20260809 07:33:05.066856 651 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.107131 30899 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:33:05.108599 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.123505 30909 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:05.096427 937 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.099172 897 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.101142 650 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:05.127293 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.144325 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:05.110360 648 consensus_meta.cc:287] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 1.145s user 0.014s sys 0.000s
I20260809 07:33:05.091101 825 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.132445 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:05.128757 768 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.167s user 0.026s sys 0.026s
I20260809 07:33:05.135576 804 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:05.148095 651 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.141175 30906 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
W20260809 07:33:05.141047 30899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 868) took 1771 ms (client timeout 2165 ms). Trace:
I20260809 07:33:05.103376 806 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.111529 30898 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
W20260809 07:33:05.442015 30898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 654) took 2730 ms (client timeout 3302 ms). Trace:
I20260809 07:33:05.199697 888 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:05.708874 888 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.128895 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 879) took 1183 ms. Trace:
I20260809 07:33:05.714803 30904 rpcz_store.cc:276] 0809 07:33:03.945461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.083902 (+1138441us) service_pool.cc:227] Handling call
0809 07:33:05.128815 (+ 44913us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.179612 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 835) took 3715 ms. Trace:
I20260809 07:33:05.716462 30891 rpcz_store.cc:276] 0809 07:33:01.464239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.822389 (+358150us) service_pool.cc:227] Handling call
0809 07:33:03.953408 (+2131019us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:33:04.771322 (+817914us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:05.109151 (+337829us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.179362 (+ 70211us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54003ad9d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.179521 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1869424256}
I20260809 07:33:05.301759 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 867) took 1941 ms. Trace:
I20260809 07:33:05.722465 30906 rpcz_store.cc:276] 0809 07:33:03.360485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.108323 (+1747838us) service_pool.cc:227] Handling call
0809 07:33:05.301679 (+193356us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.723462 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.724169 30906 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:05.724562 30906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 677) took 1763 ms (client timeout 2165 ms). Trace:
W20260809 07:33:05.724874 30906 rpcz_store.cc:269] 0809 07:33:03.961085 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.723173 (+1762088us) service_pool.cc:227] Handling call
0809 07:33:05.724514 (+ 1341us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.725638 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:05.302881 30899 rpcz_store.cc:269] 0809 07:33:03.369914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.931112 (+1561198us) service_pool.cc:227] Handling call
0809 07:33:05.140988 (+209876us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.336669 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 850) took 2034 ms. Trace:
I20260809 07:33:05.730206 31046 rpcz_store.cc:276] 0809 07:33:03.302015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.534218 (+232203us) service_pool.cc:227] Handling call
0809 07:33:03.534506 (+ 288us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:04.123157 (+588651us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.336396 (+1213239us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54004006d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:05.336604 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1334351232}
I20260809 07:33:05.731411 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.732514 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.733162 31046 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:05.733783 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.155530 31048 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:05.170634 31049 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:05.739499 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.740002 31049 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.151641 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 861) took 2308 ms. Trace:
I20260809 07:33:05.743338 31051 rpcz_store.cc:276] 0809 07:33:02.843471 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.843579 (+ 108us) service_pool.cc:227] Handling call
0809 07:33:02.857717 (+ 14138us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:02.859961 (+ 2244us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:02.859990 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:04.569072 (+1709082us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b18002dcd60. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a
0809 07:33:04.569686 (+ 614us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b2c00000238. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:33:04.570167 (+ 481us) log.cc:842] Serialized 26 byte log entry
0809 07:33:05.067332 (+497165us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:05.067365 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.067435 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:05.151401 (+ 83966us) raft_consensus.cc:1671] finished
0809 07:33:05.151443 (+ 42us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:05.151578 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1836,"raft.queue_time_us":3241,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":4390,"spinlock_wait_cycles":15848064,"thread_start_us":495041,"threads_started":1,"wal-append.queue_time_us":538214}
I20260809 07:33:05.745111 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.745608 31051 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:05.235352 31058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 885) took 2002 ms (client timeout 2497 ms). Trace:
I20260809 07:33:05.187305 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 870) took 1263 ms. Trace:
W20260809 07:33:05.747169 31058 rpcz_store.cc:269] 0809 07:33:03.233084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.262384 (+ 29300us) service_pool.cc:227] Handling call
0809 07:33:05.235139 (+1972755us) spinlock_profiling.cc:243] Waited 63.8 ms on lock 0x7b540033a9d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.235288 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":146823040}
I20260809 07:33:05.146739 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.748698 31055 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:05.747721 31059 rpcz_store.cc:276] 0809 07:33:03.923801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.040703 (+116902us) service_pool.cc:227] Handling call
0809 07:33:04.041044 (+ 341us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:04.041358 (+ 314us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:04.041382 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:04.042797 (+ 1415us) log.cc:842] Serialized 26 byte log entry
0809 07:33:05.183527 (+1140730us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b18000685c0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411 00007f4094ecd5cf
0809 07:33:05.186821 (+ 3294us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:05.186854 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.186930 (+ 76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:05.186977 (+ 47us) raft_consensus.cc:1671] finished
0809 07:33:05.186989 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:05.187219 (+ 230us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10571,"raft.queue_time_us":61,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":18931,"spinlock_wait_cycles":11174272,"thread_start_us":3126,"threads_started":1,"wal-append.queue_time_us":1123482}
W20260809 07:33:05.257939 31118 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items.
I20260809 07:33:05.173604 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:05.753804 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 580213us
I20260809 07:33:05.755338 875 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.755997 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:05.757198 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:05.356168 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 743) took 4396 ms. Trace:
I20260809 07:33:05.778040 31164 rpcz_store.cc:276] 0809 07:33:00.959681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:01.115536 (+155855us) service_pool.cc:227] Handling call
0809 07:33:01.115963 (+ 427us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:02.834992 (+1719029us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.355797 (+2520805us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b5400184f54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:05.356059 (+ 262us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3944701312}
I20260809 07:33:05.213167 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 800) took 1579 ms. Trace:
I20260809 07:33:05.780076 31165 rpcz_store.cc:276] 0809 07:33:03.634168 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.634302 (+ 134us) service_pool.cc:227] Handling call
0809 07:33:05.212980 (+1578678us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54003a02d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.213108 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1876043648}
I20260809 07:33:05.303787 31170 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:05.201979 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.203681 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.206605 31175 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:05.233108 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 807) took 1241 ms. Trace:
I20260809 07:33:05.205019 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.267566 31117 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 738)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 755)
I20260809 07:33:05.339818 32596 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.792855 370 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:05.148566 32703 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.172600 32609 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.360281 357 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:05.797345 357 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:05.786731 31181 rpcz_store.cc:276] 0809 07:33:03.991922 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.115126 (+123204us) service_pool.cc:227] Handling call
0809 07:33:04.115550 (+ 424us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:05.128333 (+1012783us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.232831 (+104498us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54000c5154. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:05.233034 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2235512192}
I20260809 07:33:05.798233 357 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.799299 357 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.804670 664 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.805179 664 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.805706 888 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.164965 32566 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.140319 32717 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.181162 388 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:33:05.829829 388 tablet_bootstrap.cc:492] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Bootstrap complete.
I20260809 07:33:05.131296 510 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.751840 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 738)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 755)
I20260809 07:33:05.139779 535 consensus_queue.cc:237] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.836324 32717 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.836814 32717 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:05.254459 570 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.165767 597 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.838500 570 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.838826 32717 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.839740 32717 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:05.840227 597 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.840402 570 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.161281 650 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:05.841653 30906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 671) took 1778 ms (client timeout 2194 ms). Trace:
I20260809 07:33:05.258060 630 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:05.843086 630 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.844702 630 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.148658 651 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:05.109664 568 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:05.196816 710 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:05.109215 675 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.898998 675 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.154451 699 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.254159 756 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.197213 768 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.295048 802 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.148414 804 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:05.907629 804 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.908373 570 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:05.908738 570 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.910125 570 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.910491 32596 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.910933 32596 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.912675 32596 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.197956 807 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:05.304392 806 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.139088 823 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.154132 825 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.145634 829 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.914219 807 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.923604 829 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.925369 807 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.925796 829 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.125901 839 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.927656 823 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.928231 839 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.199149 859 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:05.929217 859 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.929647 823 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.265795 876 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:05.930985 859 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.931131 876 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.932078 829 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.932510 829 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.932891 876 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:05.442663 30898 rpcz_store.cc:269] 0809 07:33:02.712019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.757177 (+2045158us) service_pool.cc:227] Handling call
0809 07:33:05.441917 (+684740us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.933869 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:05.934360 829 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.934476 30898 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:33:05.126176 30896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 648) took 3096 ms (client timeout 1565 ms). Trace:
I20260809 07:33:05.244047 898 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.126601 906 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.945506 906 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.197547 922 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:05.127084 936 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.946480 922 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.254876 939 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:05.152458 940 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260809 07:33:05.155261 946 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260809 07:33:05.153123 949 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.253821 951 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:05.952711 951 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.954137 951 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:05.471592 30900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 861) took 2112 ms (client timeout 1932 ms). Trace:
I20260809 07:33:05.498736 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:05.501096 603 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:05.954567 30900 rpcz_store.cc:269] 0809 07:33:03.358730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:03.457343 (+ 98613us) service_pool.cc:227] Handling call
0809 07:33:05.471376 (+2014033us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54003c2dd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.471520 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1338201344}
I20260809 07:33:05.954846 603 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.955700 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.956395 30900 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:05.956753 30900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 863) took 2597 ms (client timeout 3320 ms). Trace:
I20260809 07:33:05.956817 603 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.501742 30853 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:05.957089 30900 rpcz_store.cc:269] 0809 07:33:03.359279 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.955448 (+2596169us) service_pool.cc:227] Handling call
0809 07:33:05.956702 (+ 1254us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:05.957723 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 456026us
I20260809 07:33:05.957908 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.580987 698 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:02.599205 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:05.620733 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 841) took 4155 ms. Trace:
I20260809 07:33:05.959836 30909 rpcz_store.cc:276] 0809 07:33:01.465683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.632401 (+1166718us) service_pool.cc:227] Handling call
0809 07:33:05.123234 (+2490833us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:05.141081 (+ 17847us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.620463 (+479382us) spinlock_profiling.cc:243] Waited 2.47 s on lock 0x7b54002512d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.620646 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5677508864}
W20260809 07:33:05.960284 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 461633us
I20260809 07:33:05.961047 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.961730 30909 raft_consensus.cc:2379] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:05.962069 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 668) took 2069 ms. Trace:
I20260809 07:33:05.962343 30909 rpcz_store.cc:276] 0809 07:33:03.892479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.960753 (+2068274us) service_pool.cc:227] Handling call
0809 07:33:05.962021 (+ 1268us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.625499 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 876) took 1699 ms. Trace:
I20260809 07:33:05.963378 31045 rpcz_store.cc:276] 0809 07:33:03.926000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.617036 (+691036us) service_pool.cc:227] Handling call
0809 07:33:04.638027 (+ 20991us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:05.081427 (+443400us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.625196 (+543769us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b5400344254. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:05.625434 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":971096832}
I20260809 07:33:05.962575 949 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:05.964411 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.965581 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.966614 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.649196 482 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.672638 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 874) took 1747 ms. Trace:
I20260809 07:33:05.750102 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 888) took 1631 ms. Trace:
I20260809 07:33:05.757506 827 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:05.633523 773 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:05.759724 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.763653 875 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:05.758720 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:05.767105 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:05.772619 855 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:05.765570 602 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.717953 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.808228 32703 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:05.807351 664 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.824847 31058 raft_consensus.cc:3032] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:05.827565 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:05.828673 31006 leader_election.cc:334] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Timed out: RequestConsensusVote RPC to 127.28.87.67:44107 timed out after 1.791s (SENT)
I20260809 07:33:05.968853 31006 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:05.969022 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 202001us
I20260809 07:33:05.831614 388 ts_tablet_manager.cc:1397] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Time spent bootstrapping tablet: real 2.902s user 0.046s sys 0.033s
I20260809 07:33:05.967311 482 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:05.969676 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 141031us
I20260809 07:33:05.971325 482 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.967629 773 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.840158 32717 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.972868 30899 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:05.973657 773 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:05.842031 30906 rpcz_store.cc:269] 0809 07:33:03.947763 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.725397 (+1777634us) service_pool.cc:227] Handling call
0809 07:33:05.726288 (+ 891us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.162123 655 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.848126 31046 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:05.974021 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:05.833940 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 738)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 730)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 755)
I20260809 07:33:05.976053 30899 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:05.848860 510 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.875265 31049 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:05.876665 31172 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 07:33:05.880645 31051 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
W20260809 07:33:05.977293 31006 leader_election.cc:334] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Timed out: RequestConsensusVote RPC to 127.28.87.67:44107 timed out after 1.520s (SENT)
I20260809 07:33:05.977682 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:05.967262 31054 rpcz_store.cc:276] 0809 07:33:03.925277 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.613570 (+688293us) service_pool.cc:227] Handling call
0809 07:33:05.235906 (+622336us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:05.236266 (+ 360us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:05.236295 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:05.236671 (+ 376us) log.cc:842] Serialized 26 byte log entry
0809 07:33:05.668143 (+431472us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b18000f8e60. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411
0809 07:33:05.668336 (+ 193us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:05.668373 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.668451 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:05.668498 (+ 47us) raft_consensus.cc:1671] finished
0809 07:33:05.668510 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:05.672455 (+ 3945us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540033a9d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.672579 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1243,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":459,"spinlock_wait_cycles":1427606784,"thread_start_us":1299,"threads_started":1,"wal-append.queue_time_us":5137}
I20260809 07:33:05.882342 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.978382 510 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:05.887409 568 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.978833 31172 pending_rounds.cc:76] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 1
I20260809 07:33:05.891953 30742 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40564 (request call id 148) took 7325 ms. Trace:
I20260809 07:33:05.979090 31172 pending_rounds.cc:97] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:33:05.894625 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:05.979386 31172 raft_consensus.cc:2866] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:05.977113 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 755)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 758)
I20260809 07:33:05.979641 31165 raft_consensus.cc:2362] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4. Re-sending same reply.
I20260809 07:33:05.980410 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.896536 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.976487 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 884) took 1153 ms. Trace:
I20260809 07:33:05.900702 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.981400 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 771) took 3276 ms. Trace:
I20260809 07:33:05.981325 30899 rpcz_store.cc:276] 0809 07:33:04.823193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.973695 (+1150502us) service_pool.cc:227] Handling call
0809 07:33:05.976440 (+ 2745us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.903095 756 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.903643 768 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.968005 855 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.982458 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.982967 31054 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.980763 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 725)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 755)
I20260809 07:33:05.983470 768 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.904209 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.984025 855 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.967486 827 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:05.984747 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.905171 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:05.905836 802 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.984772 827 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:05.985594 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.926577 31057 raft_consensus.cc:2396] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:05.978606 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.986918 30906 raft_consensus.cc:2354] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4.
I20260809 07:33:05.987268 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 682) took 1820 ms. Trace:
W20260809 07:33:05.987977 31006 consensus_peers.cc:487] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a -> Peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107): Couldn't send request to peer 12107e38eb6d424abc4f45f16a9a1560. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items.. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:33:05.976161 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.967319 31059 rpcz_store.cc:276] 0809 07:33:04.118423 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.749299 (+1630876us) service_pool.cc:227] Handling call
0809 07:33:05.749608 (+ 309us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:05.749795 (+ 187us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:05.749819 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:05.749859 (+ 40us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:05.749885 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.749933 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:05.750044 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.989652 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.986306 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 862) took 2627 ms. Trace:
I20260809 07:33:05.990780 30899 rpcz_store.cc:276] 0809 07:33:03.359012 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.985350 (+2626338us) service_pool.cc:227] Handling call
0809 07:33:05.986260 (+ 910us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.991155 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.992470 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.993067 30899 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:05.993660 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 878) took 2051 ms. Trace:
I20260809 07:33:05.993948 30899 rpcz_store.cc:276] 0809 07:33:03.942440 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.991323 (+2048883us) service_pool.cc:227] Handling call
0809 07:33:05.993589 (+ 2266us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.994864 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:05.984354 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
W20260809 07:33:05.987551 31057 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 901) took 1851 ms (client timeout 2035 ms). Trace:
W20260809 07:33:05.997768 31057 rpcz_store.cc:269] 0809 07:33:04.136314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.212845 (+ 76531us) service_pool.cc:227] Handling call
0809 07:33:05.987394 (+1774549us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002409d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.987502 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3925275520}
I20260809 07:33:05.998714 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.999745 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.000259 31057 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:05.998293 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 737)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 746)
I20260809 07:33:06.000959 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.985098 756 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.990366 31006 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.003206 510 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:05.995606 31174 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:06.006525 872 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:06.001856 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 728)
I20260809 07:33:06.001951 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.006994 872 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.007467 31181 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:05.990186 31059 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 1.
I20260809 07:33:06.008726 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:06.009276 31059 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.009739 872 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:06.007779 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 727)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 733)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 719)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 721)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 724)
I20260809 07:33:05.995699 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 673) took 2044 ms. Trace:
I20260809 07:33:06.011046 30899 rpcz_store.cc:276] 0809 07:33:03.951027 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.994550 (+2043523us) service_pool.cc:227] Handling call
0809 07:33:05.995654 (+ 1104us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.011433 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.011976 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:06.012511 30899 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:05.987532 30906 rpcz_store.cc:276] 0809 07:33:04.166900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.978375 (+1811475us) service_pool.cc:227] Handling call
0809 07:33:05.987223 (+ 8848us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.014161 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.014832 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.015318 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.977528 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.016237 31181 raft_consensus.cc:2354] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:33:06.017045 30852 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.017792 602 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:06.018205 655 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:06.018438 602 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:05.948288 922 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:05.949119 939 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:05.958722 30853 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:05.131479 30890 rpcz_store.cc:276] 0809 07:33:03.948034 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.643053 (+695019us) service_pool.cc:227] Handling call
0809 07:33:05.080151 (+437098us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:05.992256 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.019315 602 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.020592 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.020956 939 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:06.021535 31165 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.021782 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:06.022821 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 64169.5us
I20260809 07:33:06.023295 655 raft_consensus.cc:686] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:06.023504 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:05.948226 936 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.959334 898 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:05.972596 388 raft_consensus.cc:348] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:06.011351 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45802 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 751)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
I20260809 07:33:05.966377 698 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.025296 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.026415 482 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:06.026804 482 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:06.026952 698 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:05.823524 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 809) took 1782 ms. Trace:
I20260809 07:33:06.027310 482 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:05.974185 773 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.968189 30891 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
W20260809 07:33:06.027983 30891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 684) took 1821 ms (client timeout 1900 ms). Trace:
W20260809 07:33:06.028376 30891 rpcz_store.cc:269] 0809 07:33:04.206581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.717609 (+1511028us) service_pool.cc:227] Handling call
0809 07:33:06.027925 (+310316us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.028671 655 consensus_queue.cc:237] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.029420 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:05.968462 31058 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:05.979179 30742 rpcz_store.cc:276] 0809 07:32:58.586772 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:58.591908 (+ 5136us) service_pool.cc:227] Handling call
0809 07:33:05.891861 (+7299953us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:00.303180 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:00.303735 (+ 555us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:00.303792 (+ 57us) write_op.cc:435] Acquired schema lock
0809 07:33:00.303804 (+ 12us) tablet.cc:660] Decoding operations
0809 07:33:00.304772 (+ 968us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:00.304823 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:00.304846 (+ 23us) tablet.cc:683] Acquiring locks for 9 operations
0809 07:33:00.305488 (+ 642us) tablet.cc:699] Row locks acquired
0809 07:33:00.305507 (+ 19us) write_op.cc:260] PREPARE: finished
0809 07:33:00.305636 (+ 129us) write_op.cc:270] Start()
0809 07:33:00.305754 (+ 118us) write_op.cc:276] Timestamp: P: 1786260780305595 usec, L: 0
0809 07:33:00.305774 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 07:33:00.311261 (+ 5487us) log.cc:842] Serialized 3614 byte log entry
0809 07:33:00.593361 (+282100us) op_driver.cc:444] REPLICATION: finished
0809 07:33:00.734920 (+141559us) write_op.cc:301] APPLY: starting
0809 07:33:00.736740 (+ 1820us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:00.736978 (+ 238us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:00.747620 (+ 10642us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:00.884968 (+137348us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:00.885181 (+ 213us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0809 07:33:00.885225 (+ 44us) write_op.cc:312] APPLY: finished
0809 07:33:00.885545 (+ 320us) log.cc:842] Serialized 70 byte log entry
0809 07:33:00.885718 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:00.885863 (+ 145us) write_op.cc:454] Released schema lock
0809 07:33:00.885907 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:00.895344 (+ 9437us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:00.915948 (+ 20604us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":107582,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":141301,"apply.run_cpu_time_us":4132,"apply.run_wall_time_us":208414,"num_ops":9,"prepare.queue_time_us":263430,"prepare.run_cpu_time_us":3592,"prepare.run_wall_time_us":45088,"replication_time_us":287484,"thread_start_us":294225,"threads_started":2}]]}
I20260809 07:33:06.030921 31058 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.032063 31058 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:05.974879 664 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:06.015420 30906 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:06.034504 664 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:06.034897 664 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:06.027536 31170 rpcz_store.cc:276] 0809 07:33:04.041617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.125776 (+ 84159us) service_pool.cc:227] Handling call
0809 07:33:04.126111 (+ 335us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:05.799831 (+1673720us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.823300 (+ 23469us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002a21d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:05.823468 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2707812736}
I20260809 07:33:06.035943 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.037146 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:05.963167 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.038605 30909 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:06.039083 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 885) took 1215 ms. Trace:
I20260809 07:33:06.022409 31172 rpcz_store.cc:276] 0809 07:33:02.704649 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.752931 (+ 48282us) service_pool.cc:227] Handling call
0809 07:33:02.753399 (+ 468us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:05.979956 (+3226557us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:05.981217 (+ 1261us) spinlock_profiling.cc:243] Waited 3.12 s on lock 0x7b54003fd9d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:05.981357 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7182096256}
I20260809 07:33:06.039345 30909 rpcz_store.cc:276] 0809 07:33:04.823533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.962914 (+1139381us) service_pool.cc:227] Handling call
0809 07:33:06.039033 (+ 76119us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.039958 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.974536 31048 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:05.977794 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:05.848385 31055 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:06.044664 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 917) took 1374 ms. Trace:
I20260809 07:33:06.044965 31055 rpcz_store.cc:276] 0809 07:33:04.669888 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.146542 (+476654us) service_pool.cc:227] Handling call
0809 07:33:06.044605 (+898063us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.045779 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.046450 31055 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.047152 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.048099 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.048887 31055 raft_consensus.cc:2396] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:06.049530 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:06.050019 31055 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 5
I20260809 07:33:05.981767 31182 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.051126 30891 raft_consensus.cc:1226] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:05.884086 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.051831 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.052404 31182 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:06.052673 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 850) took 4026 ms. Trace:
I20260809 07:33:05.902406 699 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.053148 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.042181 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:06.054563 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.055296 31182 raft_consensus.cc:2354] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5.
I20260809 07:33:05.977044 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.045116 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.052425 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.056056 31049 raft_consensus.cc:2379] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:06.056774 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:05.941128 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.056591 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.058000 31177 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:06.058099 31051 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:33:06.057382 31049 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.056634 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.059150 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:05.942543 30896 rpcz_store.cc:269] 0809 07:33:02.029339 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:02.093413 (+ 64074us) service_pool.cc:227] Handling call
0809 07:33:05.119719 (+3026306us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b54002512d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:05.124961 (+ 5242us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9148160}
I20260809 07:33:06.059798 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.060187 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.060580 31164 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:06.060870 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 851) took 4030 ms. Trace:
I20260809 07:33:06.061237 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.061374 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.943702 30898 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:06.038465 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:05.975584 31045 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.075067 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:06.055248 30906 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.075596 31045 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.039700 482 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.077585 482 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.078801 30911 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.071439 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.079716 31164 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:06.080026 854 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:06.074003 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 882) took 1461 ms. Trace:
I20260809 07:33:06.041743 30900 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.041623 823 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 07:33:06.079842 30911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 875) took 2150 ms (client timeout 1876 ms). Trace:
W20260809 07:33:06.081382 30911 rpcz_store.cc:269] 0809 07:33:03.929906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:04.687381 (+757475us) service_pool.cc:227] Handling call
0809 07:33:06.079693 (+1392312us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001937d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:06.079785 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2335042304}
I20260809 07:33:06.024732 30904 raft_consensus.cc:2379] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.076489 30852 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:06.082352 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:06.082885 30911 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.083118 823 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:06.083282 30852 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.041193 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.038388 30899 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:06.084263 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 883) took 1471 ms. Trace:
I20260809 07:33:06.040565 773 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:05.974021 32717 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:06.082804 30904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 683) took 1876 ms (client timeout 2183 ms). Trace:
I20260809 07:33:06.085031 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.085263 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.085793 30899 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.085443 823 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:06.085444 31046 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.086606 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.087318 30899 raft_consensus.cc:2379] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.088203 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:06.089224 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.061930 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.053030 30891 rpcz_store.cc:276] 0809 07:33:02.026585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.050444 (+4023859us) service_pool.cc:227] Handling call
0809 07:33:06.050847 (+ 403us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.051548 (+ 701us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.052623 (+ 1075us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.089851 30899 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.058791 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.086634 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.090786 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.091010 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.090003 31182 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:06.091521 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.092084 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.092423 31172 raft_consensus.cc:2354] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:33:06.093062 30911 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:06.093245 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.093421 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.093982 31177 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.086434 823 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:06.094655 823 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:06.094861 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.095846 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:06.085268 30904 rpcz_store.cc:269] 0809 07:33:04.206232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.758469 (+1552237us) service_pool.cc:227] Handling call
0809 07:33:06.082651 (+324182us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b5400209254. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:06.082757 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":129535232}
I20260809 07:33:06.097530 30904 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:06.097724 31055 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:06.097913 31177 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:06.055549 699 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.098738 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.017735 31175 raft_consensus.cc:2354] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:33:06.099720 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.024616 388 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 28b9965e21374d26b0ac2014c9bafdff, State: Initialized, Role: LEARNER
I20260809 07:33:06.026772 31118 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45794 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 734)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 726)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 832)
I20260809 07:33:06.093827 31049 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:06.101675 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.101318 388 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:06.080463 854 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.043962 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.078825 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.061259 30896 rpcz_store.cc:276] 0809 07:33:02.030896 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.059992 (+4029096us) service_pool.cc:227] Handling call
0809 07:33:06.060312 (+ 320us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.060481 (+ 169us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.060507 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.060548 (+ 41us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.060581 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.060697 (+ 116us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.060815 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.001950 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.099885 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 652) took 3903 ms. Trace:
I20260809 07:33:06.091392 30891 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.099354 31177 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:06.094265 31182 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:06.066534 855 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:06.064128 30853 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:06.094099 31172 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.014961 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.103430 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.170055 31174 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.170605 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.080578 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.171176 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.171490 31182 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:06.172143 31174 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.172467 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.084676 30900 rpcz_store.cc:276] 0809 07:33:04.612989 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.957676 (+1344687us) service_pool.cc:227] Handling call
0809 07:33:06.084125 (+126449us) spinlock_profiling.cc:243] Waited 74.6 ms on lock 0x7b54003c23d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:06.084213 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":171467136}
I20260809 07:33:06.093967 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.108139 31046 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:06.102959 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.018028 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.176189 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.084300 30852 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.100457 31048 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:06.172750 30904 rpcz_store.cc:276] 0809 07:33:02.196794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.097000 (+3900206us) service_pool.cc:227] Handling call
0809 07:33:06.097256 (+ 256us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.099676 (+ 2420us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.099837 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.178064 30911 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.062027 31051 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.179378 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.180112 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.180850 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.176601 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.172551 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:06.182659 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 98417.4us
I20260809 07:33:06.183496 31058 raft_consensus.cc:1261] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 07:33:06.183641 30852 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.184013 31058 pending_rounds.cc:76] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 1
I20260809 07:33:06.176811 31049 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.185434 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.173352 31165 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.180040 30911 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:06.186750 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.187562 31165 raft_consensus.cc:2354] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5.
I20260809 07:33:06.188659 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.189482 31054 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:06.190869 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.192611 31059 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:06.196357 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.197595 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.198238 31174 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:06.199129 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.076081 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:06.174330 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 110249us
I20260809 07:33:06.171506 31164 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:06.178208 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:06.202461 31058 pending_rounds.cc:97] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:33:06.202945 31058 raft_consensus.cc:2866] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:06.203840 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 891) took 2083 ms. Trace:
I20260809 07:33:06.205003 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.181030 31051 raft_consensus.cc:2396] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:06.203047 31048 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.204823 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.205271 31058 rpcz_store.cc:276] 0809 07:33:04.120114 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.824363 (+1704249us) service_pool.cc:227] Handling call
0809 07:33:05.824724 (+ 361us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.203625 (+378901us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.203795 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.206178 31182 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:06.209666 31049 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.199783 31174 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.211304 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.212175 31170 raft_consensus.cc:2396] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:06.213433 31059 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:06.215018 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 878) took 2079 ms. Trace:
I20260809 07:33:06.216346 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.217167 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.214089 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 898) took 2092 ms. Trace:
I20260809 07:33:06.218420 31059 rpcz_store.cc:276] 0809 07:33:04.121912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.212710 (+2090798us) service_pool.cc:227] Handling call
0809 07:33:06.213003 (+ 293us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.213859 (+ 856us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.214043 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.215395 31055 rpcz_store.cc:276] 0809 07:33:04.135565 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.214213 (+2078648us) service_pool.cc:227] Handling call
0809 07:33:06.214543 (+ 330us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.214733 (+ 190us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.214759 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.214799 (+ 40us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.214828 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.214875 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.214969 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.186857 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.224681 30900 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:06.225283 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 855) took 3545 ms. Trace:
I20260809 07:33:06.226554 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.227514 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.226575 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 881) took 1960 ms. Trace:
I20260809 07:33:06.227545 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 913) took 1562 ms. Trace:
I20260809 07:33:06.228305 31049 rpcz_store.cc:276] 0809 07:33:04.266247 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.225799 (+1959552us) service_pool.cc:227] Handling call
0809 07:33:06.226129 (+ 330us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.226282 (+ 153us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.226305 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.226365 (+ 60us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.226391 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.226438 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.226527 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":15232}
I20260809 07:33:06.229580 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.225586 30900 rpcz_store.cc:276] 0809 07:33:02.680062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.215213 (+3535151us) service_pool.cc:227] Handling call
0809 07:33:06.215497 (+ 284us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.225054 (+ 9557us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.225230 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.228359 31057 rpcz_store.cc:276] 0809 07:33:04.664808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.226840 (+1562032us) service_pool.cc:227] Handling call
0809 07:33:06.227127 (+ 287us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.227284 (+ 157us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.227305 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.227340 (+ 35us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.227367 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.227411 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.227498 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.230746 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.227710 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.231818 31170 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.232988 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.233196 31055 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.234827 31059 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:06.100478 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.236764 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 886) took 1412 ms. Trace:
I20260809 07:33:06.255146 30852 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:06.262868 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 902) took 2121 ms. Trace:
I20260809 07:33:06.263319 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.233733 31181 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:06.255788 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.264693 31170 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.264796 31174 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.073303 664 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.225236 939 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:06.258775 31045 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.260773 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 658) took 3359 ms. Trace:
I20260809 07:33:06.195696 802 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:06.261087 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.231309 30900 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:06.261802 31048 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:06.254566 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 885) took 1431 ms. Trace:
I20260809 07:33:06.260842 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.276254 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 880) took 2041 ms. Trace:
I20260809 07:33:06.263765 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.254072 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.276569 31048 rpcz_store.cc:276] 0809 07:33:04.234478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.214449 (+1979971us) service_pool.cc:227] Handling call
0809 07:33:06.261481 (+ 47032us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.276027 (+ 14546us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.276204 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.258158 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.277434 31165 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.263267 31051 rpcz_store.cc:276] 0809 07:33:04.141040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.214388 (+2073348us) service_pool.cc:227] Handling call
0809 07:33:06.262420 (+ 48032us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.262578 (+ 158us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.262603 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.262642 (+ 39us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.262669 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.262714 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.262812 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.267167 31057 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 5. Ops: []
I20260809 07:33:06.220386 603 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:33:06.269867 30911 rpcz_store.cc:276] 0809 07:33:02.901120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.233551 (+3332431us) service_pool.cc:227] Handling call
0809 07:33:06.233996 (+ 445us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.260439 (+ 26443us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.260469 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.260511 (+ 42us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.260540 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.260604 (+ 64us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.260715 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26123,"raft.queue_time_us":30069}
I20260809 07:33:06.278095 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.278689 30900 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:06.278793 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.265542 31049 rpcz_store.cc:276] 0809 07:33:04.823897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.236033 (+1412136us) service_pool.cc:227] Handling call
0809 07:33:06.236330 (+ 297us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.236488 (+ 158us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.236509 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.236547 (+ 38us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.236573 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.236620 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.236710 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.276973 31055 rpcz_store.cc:276] 0809 07:33:04.823618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.232778 (+1409160us) service_pool.cc:227] Handling call
0809 07:33:06.233042 (+ 264us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.254332 (+ 21290us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.254513 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.277765 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.235383 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.275048 31045 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:06.282786 31049 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:06.284085 31174 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:06.269426 664 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:06.267567 31169 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.253655 634 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:06.291118 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:06.282019 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 59116.9us
I20260809 07:33:06.291785 31181 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.284916 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 723) took 1074 ms. Trace:
I20260809 07:33:06.286167 31057 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:06.275735 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.292661 31172 rpcz_store.cc:276] 0809 07:33:05.210632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.200877 (+990245us) service_pool.cc:227] Handling call
0809 07:33:06.284835 (+ 83958us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.293835 31170 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.294641 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.305989 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 729) took 1093 ms. Trace:
I20260809 07:33:06.269784 655 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:06.288483 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.287256 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 884) took 1465 ms. Trace:
I20260809 07:33:06.310263 30911 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.310559 31059 rpcz_store.cc:276] 0809 07:33:04.821926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.234258 (+1412332us) service_pool.cc:227] Handling call
0809 07:33:06.234603 (+ 345us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.287005 (+ 52402us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.287199 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.165092 827 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.315215 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.279449 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:06.226701 30891 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:06.286507 603 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:06.293956 802 raft_consensus.cc:686] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:06.291517 939 raft_consensus.cc:686] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:06.292156 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.309408 31048 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.311264 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.311540 388 ts_tablet_manager.cc:1428] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Time spent starting tablet: real 0.342s user 0.010s sys 0.025s
I20260809 07:33:06.169863 30898 rpcz_store.cc:276] 0809 07:33:04.612709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:05.933595 (+1320886us) service_pool.cc:227] Handling call
0809 07:33:06.073947 (+140352us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.327803 603 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:06.316458 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.330605 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.320180 31005 leader_election.cc:398] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4.
I20260809 07:33:06.331152 31005 leader_election.cc:401] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:06.332669 30900 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.290143 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 736) took 1075 ms. Trace:
I20260809 07:33:06.322329 31006 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.307986 30896 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.323056 31172 raft_consensus.cc:2354] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5.
I20260809 07:33:06.290577 936 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:06.306847 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.323061 634 raft_consensus.cc:686] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:06.291273 602 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:06.325591 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.336127 31006 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.284566 31177 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.328456 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.308462 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.289639 872 consensus_queue.cc:1046] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260809 07:33:06.335240 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:06.306344 31170 rpcz_store.cc:276] 0809 07:33:05.212273 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.275489 (+1063216us) service_pool.cc:227] Handling call
0809 07:33:06.305925 (+ 30436us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.333858 31181 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:06.323326 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.338088 30911 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:06.337699 31006 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.334393 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.345212 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.336186 31182 rpcz_store.cc:276] 0809 07:33:05.214623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.232160 (+1017537us) service_pool.cc:227] Handling call
0809 07:33:06.290083 (+ 57923us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.345278 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.311973 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.344612 31165 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:06.345590 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.346984 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.348053 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 713) took 1140 ms. Trace:
I20260809 07:33:06.323047 839 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260809 07:33:06.356629 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.348205 31165 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.348881 31174 rpcz_store.cc:276] 0809 07:33:05.207795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.347263 (+1139468us) service_pool.cc:227] Handling call
0809 07:33:06.347614 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.347777 (+ 163us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.347800 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.347836 (+ 36us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.347861 (+ 25us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.347906 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.348003 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.345872 30911 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:06.357468 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 715) took 1149 ms. Trace:
I20260809 07:33:06.350641 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.351850 31170 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:33:06.352382 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 812) took 1095 ms. Trace:
I20260809 07:33:06.358628 31177 rpcz_store.cc:276] 0809 07:33:05.256676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.260758 (+1004082us) service_pool.cc:227] Handling call
0809 07:33:06.352322 (+ 91564us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.359001 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:06.360236 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 717) took 1151 ms. Trace:
I20260809 07:33:06.361898 31177 rpcz_store.cc:276] 0809 07:33:05.208884 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.359279 (+1150395us) service_pool.cc:227] Handling call
0809 07:33:06.359780 (+ 501us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.359949 (+ 169us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.359974 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.360014 (+ 40us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.360041 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.360088 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.360183 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.361497 31172 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:06.340552 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 48472.5us
I20260809 07:33:06.357795 31165 rpcz_store.cc:276] 0809 07:33:05.208346 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.347796 (+1139450us) service_pool.cc:227] Handling call
0809 07:33:06.348052 (+ 256us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.357262 (+ 9210us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.357422 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.364789 31165 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.365414 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 722) took 1155 ms. Trace:
I20260809 07:33:06.365770 31165 rpcz_store.cc:276] 0809 07:33:05.210380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.364344 (+1153964us) service_pool.cc:227] Handling call
0809 07:33:06.364625 (+ 281us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.365195 (+ 570us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.365356 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.362764 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 718) took 1153 ms. Trace:
I20260809 07:33:06.369892 31182 rpcz_store.cc:276] 0809 07:33:05.209150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.362034 (+1152884us) service_pool.cc:227] Handling call
0809 07:33:06.362289 (+ 255us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.362493 (+ 204us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.362517 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.362556 (+ 39us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.362582 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.362627 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.362714 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.371876 31182 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:06.342234 31169 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 2.
I20260809 07:33:06.361312 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.325047 823 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.363026 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 716) took 1154 ms. Trace:
I20260809 07:33:06.374571 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.384121 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:06.384210 31170 rpcz_store.cc:276] 0809 07:33:05.208618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.351278 (+1142660us) service_pool.cc:227] Handling call
0809 07:33:06.351552 (+ 274us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.362764 (+ 11212us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.362969 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.343780 510 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.383261 31174 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:06.340561 773 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4.
I20260809 07:33:06.381946 31182 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:06.386123 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:06.343652 32703 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:06.393217 482 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:06.393672 31181 raft_consensus.cc:1226] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260809 07:33:06.401520 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.393702 568 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:06.396132 31006 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.354730 922 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:06.348733 936 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:06.400641 30904 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.373127 664 raft_consensus.cc:3032] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:06.360702 949 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:06.378589 829 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:06.372841 650 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:06.406905 31006 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.428746 31006 leader_election.cc:398] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:33:06.359311 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.343921 32717 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260809 07:33:06.372856 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 856) took 3691 ms. Trace:
I20260809 07:33:06.387497 876 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:06.415253 30900 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:06.385233 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.417024 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.417575 859 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:06.391480 602 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:06.389853 675 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.429291 922 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:06.401826 768 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:06.401770 939 consensus_queue.cc:237] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:06.445703 30962 debug-util.cc:398] Leaking SignalData structure 0x7b08001db9e0 after lost signal to thread 29026
W20260809 07:33:06.426662 987 log.cc:924] Time spent T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.143s user 0.001s sys 0.000s
I20260809 07:33:06.455502 876 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:06.378983 655 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260809 07:33:06.401575 482 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:06.429019 31006 leader_election.cc:401] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:06.431751 30904 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.502926 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.412055 32703 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:06.484174 675 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:06.491523 768 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:06.418540 949 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:06.460496 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 76063.4us
I20260809 07:33:06.420039 829 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.391264 510 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:06.491721 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 74763.9us
I20260809 07:33:06.547927 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.492805 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:06.489221 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 149419us
I20260809 07:33:06.426472 807 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:06.415699 664 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:06.540758 675 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.401979 802 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:06.463203 859 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.446836 30906 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:06.503479 30904 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:06.534029 31164 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:06.447700 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 911) took 1784 ms. Trace:
I20260809 07:33:06.497247 568 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:06.564687 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 653) took 3853 ms. Trace:
I20260809 07:33:06.549259 510 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.434130 634 consensus_queue.cc:237] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.448701 30896 rpcz_store.cc:276] 0809 07:33:02.681759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.227293 (+3545534us) service_pool.cc:227] Handling call
0809 07:33:06.284039 (+ 56746us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.335278 (+ 51239us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.359237 (+ 23959us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.500934 876 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.492443 30898 raft_consensus.cc:2354] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:33:06.554592 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.544361 768 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:06.546012 32703 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3.
I20260809 07:33:06.545526 949 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.552845 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.546804 829 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:33:06.552387 30962 debug-util.cc:398] Leaking SignalData structure 0x7b08000e85c0 after lost signal to thread 30803
I20260809 07:33:06.563759 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 892) took 2443 ms. Trace:
I20260809 07:33:06.562912 31046 rpcz_store.cc:276] 0809 07:33:04.663155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.225853 (+1562698us) service_pool.cc:227] Handling call
0809 07:33:06.226142 (+ 289us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.226299 (+ 157us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:06.339269 (+112970us) log.cc:842] Serialized 14 byte log entry
0809 07:33:06.447338 (+108069us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.447403 (+ 65us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.447445 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.447509 (+ 64us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.447641 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16542,"prepare.queue_time_us":177440,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":75394,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4090,"spinlock_wait_cycles":30208,"thread_start_us":163858,"threads_started":2,"wal-append.queue_time_us":158433}
I20260809 07:33:06.565090 30906 rpcz_store.cc:276] 0809 07:33:02.711681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.215222 (+3503541us) service_pool.cc:227] Handling call
0809 07:33:06.446116 (+230894us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b5800001898. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:33:06.446365 (+ 249us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.564405 (+118040us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.564626 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8559360}
I20260809 07:33:06.492813 827 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:06.497879 936 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.551179 807 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:06.612258 31172 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:06.591097 30911 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:06.577247 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 148608us
I20260809 07:33:06.568325 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.564373 906 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:06.600708 31054 rpcz_store.cc:276] 0809 07:33:04.120379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.215005 (+2094626us) service_pool.cc:227] Handling call
0809 07:33:06.230943 (+ 15938us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.231139 (+ 196us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:06.405785 (+174646us) log.cc:842] Serialized 14 byte log entry
0809 07:33:06.540374 (+134589us) log.cc:842] Serialized 14 byte log entry
0809 07:33:06.563343 (+ 22969us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.563390 (+ 47us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.563429 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.563602 (+ 173us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.563710 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":36830,"prepare.queue_time_us":184642,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":1677,"raft.queue_time_us":12996,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2048,"spinlock_wait_cycles":29594624,"thread_start_us":272587,"threads_started":3,"wal-append.queue_time_us":161595}
I20260809 07:33:06.616343 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.565198 859 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:06.555753 872 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
W20260809 07:33:06.612259 30962 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8500 after lost signal to thread 30827
I20260809 07:33:06.591282 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.613996 31006 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.615527 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.600517 664 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.621069 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:06.550923 602 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.622393 875 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.629280 875 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.629743 875 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.628903 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 849) took 4604 ms. Trace:
I20260809 07:33:06.613663 568 raft_consensus.cc:686] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:06.627210 31006 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.628293 872 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:06.616624 823 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.628679 1017 raft_consensus.cc:2741] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 07:33:06.632089 30911 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.632061 30909 rpcz_store.cc:276] 0809 07:33:02.024489 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.040005 (+4015516us) service_pool.cc:227] Handling call
0809 07:33:06.040296 (+ 291us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.040485 (+ 189us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:33:06.307444 (+266959us) log.cc:842] Serialized 14 byte log entry
0809 07:33:06.628433 (+320989us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b1800370360. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c990 00007f4091fce864 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7 00007f4094f14058
0809 07:33:06.628566 (+ 133us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.628609 (+ 43us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.628647 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.628710 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.628832 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12424,"prepare.queue_time_us":309309,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":3793,"raft.queue_time_us":252589,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":27817728,"thread_start_us":337063,"threads_started":2,"wal-append.queue_time_us":109938}
I20260809 07:33:06.643869 30911 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 07:33:06.634171 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 81408.7us
I20260809 07:33:06.636210 31006 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.635592 650 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:06.638501 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 93207.5us
W20260809 07:33:06.638939 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 89843.7us
I20260809 07:33:06.617808 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:06.629444 998 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
I20260809 07:33:06.568323 922 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.644490 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 663) took 3514 ms. Trace:
I20260809 07:33:06.648722 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:06.626720 30891 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.648063 31175 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:06.649848 30911 rpcz_store.cc:276] 0809 07:33:03.130052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.643400 (+3513348us) service_pool.cc:227] Handling call
0809 07:33:06.643612 (+ 212us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.644255 (+ 643us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.644438 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.645041 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.651850 31006 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.655586 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.657152 31006 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.658535 30911 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.658603 31006 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.659864 32703 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:06.660249 32703 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:06.660617 32703 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:06.635687 30898 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:06.662632 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 857) took 3361 ms. Trace:
I20260809 07:33:06.667076 30911 rpcz_store.cc:276] 0809 07:33:03.301705 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.653296 (+3351591us) service_pool.cc:227] Handling call
0809 07:33:06.653561 (+ 265us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.662400 (+ 8839us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.662581 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.667405 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.668709 31006 leader_election.cc:398] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5.
I20260809 07:33:06.668998 31006 leader_election.cc:401] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:06.641618 1007 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.666265 1017 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.672389 1017 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:06.673007 773 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.673381 773 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.673707 773 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.674204 854 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:06.674396 30962 debug-util.cc:398] Leaking SignalData structure 0x7b08003a90a0 after lost signal to thread 30977
I20260809 07:33:06.670074 32717 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:06.675278 32717 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5.
I20260809 07:33:06.672696 1007 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.677259 1007 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.672766 1017 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.660151 30891 raft_consensus.cc:1261] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:06.686650 30853 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:06.675493 30962 debug-util.cc:398] Leaking SignalData structure 0x7b08003a98c0 after lost signal to thread 31100
W20260809 07:33:06.672531 1003 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
W20260809 07:33:06.680045 997 log.cc:924] Time spent T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.180s user 0.001s sys 0.000s
I20260809 07:33:06.686666 30896 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.694334 30853 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:06.649251 698 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260809 07:33:06.656795 1014 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:06.700519 31006 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:06.680207 1024 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.705134 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:06.666534 823 consensus_queue.cc:1046] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260809 07:33:06.697317 827 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:06.698915 597 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:06.644497 807 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.689944 1023 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:06.686236 855 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260809 07:33:06.698177 568 consensus_queue.cc:237] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.708151 32717 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.703884 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 858) took 3400 ms. Trace:
I20260809 07:33:06.706733 1024 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:06.704607 1014 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.725430 675 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.726203 32703 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.729313 875 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.714349 898 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:06.730499 807 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.708415 650 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.715915 827 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.715603 854 consensus_queue.cc:1046] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260809 07:33:06.722190 32717 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:06.697575 1002 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260809 07:33:06.715909 597 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:06.719796 30962 debug-util.cc:398] Leaking SignalData structure 0x7b08002f69c0 after lost signal to thread 31219
W20260809 07:33:06.802330 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 108052us
I20260809 07:33:06.722539 1023 raft_consensus.cc:686] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:06.822983 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 113859us
I20260809 07:33:06.783495 1024 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:06.791922 997 log.cc:924] Time spent T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260809 07:33:06.782656 1014 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.741326 603 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.827899 30853 leader_election.cc:398] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:33:06.835007 30853 leader_election.cc:401] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260809 07:33:06.827296 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 136435us
I20260809 07:33:06.829178 1034 consensus_queue.cc:1046] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:06.703657 872 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:06.842337 1007 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.844254 1007 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.789578 773 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.789008 1029 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:06.850821 1029 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:06.851207 1029 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.851641 859 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.741842 30891 rpcz_store.cc:276] 0809 07:33:03.302068 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.659384 (+3357316us) service_pool.cc:227] Handling call
0809 07:33:06.659757 (+ 373us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.689326 (+ 29569us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.702530 (+ 13204us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.860446 31049 raft_consensus.cc:1261] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:06.864786 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.865526 31049 raft_consensus.cc:2379] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:06.839535 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:06.840525 30853 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:06.841089 1037 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3.
I20260809 07:33:06.847007 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:06.849756 31051 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:06.852007 773 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.823865 32717 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:06.855019 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 664) took 3697 ms. Trace:
I20260809 07:33:06.874893 30909 rpcz_store.cc:276] 0809 07:33:03.157706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.644217 (+3486511us) service_pool.cc:227] Handling call
0809 07:33:06.662959 (+ 18742us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.663210 (+ 251us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.663226 (+ 16us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:06.663507 (+ 281us) log.cc:842] Serialized 26 byte log entry
0809 07:33:06.824453 (+160946us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.854540 (+ 30087us) log.cc:842] Serialized 14 byte log entry
0809 07:33:06.854737 (+ 197us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.854832 (+ 95us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:06.854867 (+ 35us) raft_consensus.cc:1671] finished
0809 07:33:06.854877 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.854972 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11684,"prepare.queue_time_us":12315,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":4051,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":4707,"thread_start_us":127326,"threads_started":1,"wal-append.queue_time_us":167604}
I20260809 07:33:06.876981 30909 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.877501 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 877) took 2941 ms. Trace:
I20260809 07:33:06.877799 30909 rpcz_store.cc:276] 0809 07:33:03.935700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.876505 (+2940805us) service_pool.cc:227] Handling call
0809 07:33:06.876823 (+ 318us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.877308 (+ 485us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.877453 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.827088 597 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:06.859043 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:06.883884 30909 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:06.888780 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 669) took 2941 ms. Trace:
I20260809 07:33:06.889178 30909 rpcz_store.cc:276] 0809 07:33:03.947036 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.883397 (+2936361us) service_pool.cc:227] Handling call
0809 07:33:06.883710 (+ 313us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.888508 (+ 4798us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.888717 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.890761 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 670) took 2943 ms. Trace:
I20260809 07:33:06.891127 30909 rpcz_store.cc:276] 0809 07:33:03.947365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.889960 (+2942595us) service_pool.cc:227] Handling call
0809 07:33:06.890282 (+ 322us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.890470 (+ 188us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.890495 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.890534 (+ 39us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.890562 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.890607 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.890704 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.892776 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 674) took 2941 ms. Trace:
I20260809 07:33:06.893147 30909 rpcz_store.cc:276] 0809 07:33:03.951341 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.892041 (+2940700us) service_pool.cc:227] Handling call
0809 07:33:06.892342 (+ 301us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.892500 (+ 158us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.892523 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.892560 (+ 37us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.892586 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.892630 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.892724 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.894711 30909 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:33:06.895313 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 675) took 2943 ms. Trace:
I20260809 07:33:06.887639 597 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.897184 597 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.897729 30909 rpcz_store.cc:276] 0809 07:33:03.951805 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.894065 (+2942260us) service_pool.cc:227] Handling call
0809 07:33:06.894398 (+ 333us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.895096 (+ 698us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.895268 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.899314 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 880) took 2764 ms. Trace:
I20260809 07:33:06.900696 30909 rpcz_store.cc:276] 0809 07:33:04.134909 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.898580 (+2763671us) service_pool.cc:227] Handling call
0809 07:33:06.898894 (+ 314us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.899047 (+ 153us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.899070 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.899108 (+ 38us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.899136 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.899180 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.899272 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.903548 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 680) took 2757 ms. Trace:
I20260809 07:33:06.905117 30909 rpcz_store.cc:276] 0809 07:33:04.146543 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.902826 (+2756283us) service_pool.cc:227] Handling call
0809 07:33:06.903132 (+ 306us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.903287 (+ 155us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.903311 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:06.903349 (+ 38us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:06.903375 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.903418 (+ 43us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.903506 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.907243 30909 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:06.908773 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 681) took 2742 ms. Trace:
I20260809 07:33:06.909704 30909 rpcz_store.cc:276] 0809 07:33:04.166500 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.906783 (+2740283us) service_pool.cc:227] Handling call
0809 07:33:06.907092 (+ 309us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.908558 (+ 1466us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.908722 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.911558 30909 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:06.913407 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 887) took 1895 ms. Trace:
I20260809 07:33:06.914300 30909 rpcz_store.cc:276] 0809 07:33:05.018052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.911015 (+1892963us) service_pool.cc:227] Handling call
0809 07:33:06.911345 (+ 330us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.913192 (+ 1847us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.913367 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:06.916535 30909 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:06.917104 1029 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.919828 30909 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:06.854069 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 874) took 3234 ms. Trace:
I20260809 07:33:06.947949 30904 raft_consensus.cc:2379] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:06.861929 1022 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.205s user 0.001s sys 0.000s
I20260809 07:33:06.853562 859 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:06.952944 1011 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.282s user 0.000s sys 0.001s
I20260809 07:33:06.824783 510 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:06.960197 31045 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:06.948287 675 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:06.959923 768 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.002033 949 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.009213 875 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:07.011794 876 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.020347 876 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.022008 949 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.023341 1029 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:07.024192 1032 log.cc:924] Time spent T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.235s user 0.001s sys 0.000s
I20260809 07:33:06.989882 1023 consensus_queue.cc:237] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.034711 1017 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.036935 1017 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.072270 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 661) took 4164 ms. Trace:
I20260809 07:33:07.073603 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 872) took 3466 ms. Trace:
I20260809 07:33:07.073468 30890 rpcz_store.cc:276] 0809 07:33:02.907993 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.629840 (+3721847us) service_pool.cc:227] Handling call
0809 07:33:06.630171 (+ 331us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.631148 (+ 977us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:06.631456 (+ 308us) log.cc:842] Serialized 14 byte log entry
0809 07:33:07.070955 (+439499us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:07.071122 (+ 167us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:07.071263 (+ 141us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.071478 (+ 215us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:07.071854 (+ 376us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":2451,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":41,"raft.queue_time_us":12923,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3062,"thread_start_us":426782,"threads_started":2,"wal-append.queue_time_us":87888}
I20260809 07:33:06.950517 30896 rpcz_store.cc:276] 0809 07:33:03.619612 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.714638 (+3095026us) service_pool.cc:227] Handling call
0809 07:33:06.731128 (+ 16490us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.731604 (+ 476us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.731636 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:06.732171 (+ 535us) log.cc:842] Serialized 26 byte log entry
0809 07:33:06.786298 (+ 54127us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:06.786360 (+ 62us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.817306 (+ 30946us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:06.853856 (+ 36550us) raft_consensus.cc:1671] finished
0809 07:33:06.853877 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:06.854011 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1262,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":53618,"threads_started":1,"wal-append.queue_time_us":54067}
I20260809 07:33:07.074268 30911 rpcz_store.cc:276] 0809 07:33:03.607254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.676352 (+3069098us) service_pool.cc:227] Handling call
0809 07:33:06.676662 (+ 310us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.699285 (+ 22623us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.699307 (+ 22us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:06.702161 (+ 2854us) log.cc:842] Serialized 26 byte log entry
0809 07:33:07.072636 (+370475us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:07.072773 (+ 137us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.072933 (+ 160us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:07.073039 (+ 106us) raft_consensus.cc:1671] finished
0809 07:33:07.073056 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:07.073398 (+ 342us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16507,"raft.queue_time_us":16700,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3179,"thread_start_us":364058,"threads_started":1,"wal-append.queue_time_us":308074}
W20260809 07:33:06.971192 30962 debug-util.cc:398] Leaking SignalData structure 0x7b0800349120 after lost signal to thread 32268
I20260809 07:33:07.078389 30890 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:33:07.080539 30906 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:07.082154 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 650) took 5033 ms. Trace:
I20260809 07:33:07.082644 30899 rpcz_store.cc:276] 0809 07:33:02.048268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.094664 (+4046396us) service_pool.cc:227] Handling call
0809 07:33:06.094965 (+ 301us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.095350 (+ 385us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:33:06.294160 (+198810us) log.cc:842] Serialized 14 byte log entry
0809 07:33:06.631011 (+336851us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:06.631483 (+ 472us) log.cc:842] Serialized 26 byte log entry
0809 07:33:06.631600 (+ 117us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:06.631631 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.631695 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:07.081597 (+449902us) raft_consensus.cc:1671] finished
0809 07:33:07.081632 (+ 35us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:07.081919 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":244069,"prepare.queue_time_us":276974,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":243278,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":6740,"thread_start_us":265198,"threads_started":2,"wal-append.queue_time_us":117972}
W20260809 07:33:07.086457 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 245875us
I20260809 07:33:07.088320 30853 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:06.963299 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 130174us
W20260809 07:33:07.091142 31005 leader_election.cc:334] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.823s (SENT)
I20260809 07:33:07.093354 31006 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:07.104602 31051 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:07.104986 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:07.112409 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.114897 30853 leader_election.cc:398] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5.
I20260809 07:33:07.115240 30853 leader_election.cc:401] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260809 07:33:07.117272 30853 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.119192 30853 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.122135 31057 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:07.120736 1069 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:07.123338 1069 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.124682 31057 raft_consensus.cc:3032] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:07.125056 31057 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.125412 1068 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:07.126863 31057 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.128788 31057 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.139190 31172 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:33:07.139600 31175 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 07:33:07.139467 698 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:07.140599 675 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:07.139823 603 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.140970 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.141567 698 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:07.142163 675 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.142509 698 consensus_queue.cc:1046] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:07.143333 602 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:07.143879 602 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:07.143986 698 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5.
I20260809 07:33:07.140218 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.144906 698 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:07.145812 698 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:07.146220 698 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:07.147289 802 raft_consensus.cc:3032] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:06.997716 32717 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.135212 31049 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:07.150658 31175 pending_rounds.cc:76] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 1
I20260809 07:33:07.141741 603 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:07.140688 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.146654 675 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.146687 698 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:06.992784 829 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.171373 31048 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:07.172605 829 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.173128 30891 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 1. Current term is 2. Ops: []
I20260809 07:33:07.147665 32703 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.174165 876 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:07.152690 31175 pending_rounds.cc:97] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:33:07.175205 31175 raft_consensus.cc:2866] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:07.176220 876 consensus_queue.cc:1046] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:07.157441 768 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:07.161288 992 log.cc:924] Time spent T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.736s user 0.003s sys 0.015s
I20260809 07:33:07.177400 876 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:07.161307 32717 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.178943 32717 raft_consensus.cc:3032] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:07.179431 32717 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.158926 510 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.155936 603 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:07.179560 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 915) took 2513 ms. Trace:
I20260809 07:33:07.180837 32717 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.181663 936 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:07.181324 31058 rpcz_store.cc:276] 0809 07:33:04.666032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.231378 (+1565346us) service_pool.cc:227] Handling call
0809 07:33:06.231686 (+ 308us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:06.258609 (+ 26923us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:06.258640 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:06.259112 (+ 472us) log.cc:842] Serialized 26 byte log entry
0809 07:33:06.331361 (+ 72249us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:06.331408 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:06.331468 (+ 60us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:07.160093 (+828625us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:07.179030 (+ 18937us) raft_consensus.cc:1671] finished
0809 07:33:07.179053 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:07.179349 (+ 296us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13918,"raft.queue_time_us":578,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":13733,"spinlock_wait_cycles":6855424,"thread_start_us":37183,"threads_started":1,"wal-append.queue_time_us":137021}
I20260809 07:33:07.147724 802 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:07.184682 802 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.174566 32703 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:07.185879 32703 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:07.178731 1007 raft_consensus.cc:3032] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:07.150342 602 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.190464 1034 consensus_queue.cc:1225] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:07.187510 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.186519 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.180795 603 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:07.186249 802 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:07.186357 32703 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:07.191342 32717 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:07.192041 1007 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.165042 30744 catalog_manager.cc:5495] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: config changed from index -1 to 2, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added, no longer has a pending config: VOTER 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65), VOTER a720817fc113446a8ed55e61523b280a (127.28.87.66), VOTER 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67), NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:07.205431 31172 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:07.205632 482 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.206884 482 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:07.188117 664 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.211513 634 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:07.212574 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.213539 1024 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.216073 1024 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:07.217573 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.217586 31182 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:07.240208 827 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:33:07.224675 1007 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.226920 30898 raft_consensus.cc:1226] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:07.229521 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 841) took 1119 ms. Trace:
I20260809 07:33:07.241560 698 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.223093 597 raft_consensus.cc:3032] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:07.252043 698 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.252943 597 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:07.261615 1049 raft_consensus.cc:3032] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:07.262099 1049 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.262413 597 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.263542 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 659) took 4362 ms. Trace:
I20260809 07:33:07.263322 1049 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.264142 597 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:07.264866 1049 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:33:07.266445 30830 thread.cc:636] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.637s user 0.009s sys 0.014s
W20260809 07:33:07.266952 30830 thread.cc:602] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.666s user 0.010s sys 0.014s
I20260809 07:33:07.268989 32703 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.271060 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.271586 1069 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.271864 31164 raft_consensus.cc:2379] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:07.263953 30906 rpcz_store.cc:276] 0809 07:33:02.901393 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.617279 (+3715886us) service_pool.cc:227] Handling call
0809 07:33:06.627533 (+ 10254us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:07.081152 (+453619us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.263259 (+182107us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54001b3ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:07.263486 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1040388992}
I20260809 07:33:07.275144 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.275871 31174 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:07.276264 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.278154 1062 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:07.278825 829 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:07.283655 31057 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:07.245532 31177 rpcz_store.cc:276] 0809 07:33:06.109728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.368677 (+258949us) service_pool.cc:227] Handling call
0809 07:33:06.368935 (+ 258us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:06.369120 (+ 185us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:06.501698 (+132578us) log.cc:842] Serialized 14 byte log entry
0809 07:33:07.228572 (+726874us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b180037d560. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b8057b
0809 07:33:07.228905 (+ 333us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:07.229017 (+ 112us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:07.229098 (+ 81us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.229177 (+ 79us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:07.229319 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":214084,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":3714,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":167,"spinlock_wait_cycles":3862400,"thread_start_us":208142,"threads_started":2,"wal-append.queue_time_us":141200}
I20260809 07:33:07.284729 906 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:07.290771 906 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:07.294700 650 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:07.297001 698 consensus_queue.cc:1046] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:07.287328 936 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:07.298493 936 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.300985 31172 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:07.304767 854 raft_consensus.cc:3032] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:07.305243 854 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:07.307056 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.310014 32703 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.311129 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:07.335266 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:07.329826 1066 log.cc:924] Time spent T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.111s user 0.000s sys 0.004s
I20260809 07:33:07.344277 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:07.334612 1007 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:07.345875 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.346437 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:07.347321 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.348297 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:07.350178 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:07.353307 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 234327us
I20260809 07:33:07.306252 854 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.356280 854 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:07.356596 31182 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:07.357503 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:07.359161 1014 consensus_meta.cc:287] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.512s user 0.013s sys 0.000s
I20260809 07:33:07.338780 32703 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:07.324988 602 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.361157 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.383651 31051 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:07.359606 1014 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.384589 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:07.385131 829 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:07.385562 32703 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.386906 603 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.387895 31046 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:07.388775 31174 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:07.389847 30909 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260809 07:33:07.392285 906 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.394726 31169 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 3: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:07.396212 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:07.396690 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:07.383635 597 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260809 07:33:07.408959 30909 pending_rounds.cc:76] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Aborting all ops after (but not including) 2
I20260809 07:33:07.411998 30909 pending_rounds.cc:97] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Aborting uncommitted NO_OP operation due to leader change: 5.3
W20260809 07:33:07.410064 1035 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.625s user 0.001s sys 0.002s
I20260809 07:33:07.408730 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.398488 31051 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
W20260809 07:33:07.414219 807 consensus_meta.cc:287] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.621s user 0.011s sys 0.002s
I20260809 07:33:07.414839 31057 raft_consensus.cc:2396] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:07.415422 31165 raft_consensus.cc:2396] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:07.416273 31057 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:07.420799 603 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.421490 597 consensus_queue.cc:1225] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:07.423911 1014 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:07.424345 1071 log.cc:924] Time spent T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.202s user 0.001s sys 0.001s
I20260809 07:33:07.424799 31045 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:07.427112 32703 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.428243 906 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.428927 31049 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:07.412248 30909 raft_consensus.cc:2866] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:07.389004 823 raft_consensus.cc:3032] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
W20260809 07:33:07.430365 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.526s user 0.005s sys 0.004s
W20260809 07:33:07.430820 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 775210us
I20260809 07:33:07.432116 31181 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:07.434664 823 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:07.436296 698 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:07.436954 829 raft_consensus.cc:686] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.444963 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.445072 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:07.445638 31174 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:07.448513 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:07.449368 31006 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:07.450584 31181 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.453219 32199 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 3 LEARNER]: Advancing to term 5
I20260809 07:33:07.441875 936 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.460589 823 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.462553 823 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.464732 31059 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:07.458676 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:07.466794 30849 consensus_peers.cc:487] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 -> Peer 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299): Couldn't send request to peer 28b9965e21374d26b0ac2014c9bafdff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 335fccb907184174b4a6428ce262e7dc. This is attempt 1: this message will repeat every 5th retry.
W20260809 07:33:07.467191 30849 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 77594.7us
I20260809 07:33:07.467639 936 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.468495 603 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:07.468657 1054 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:07.470734 936 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:07.468690 597 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260809 07:33:07.473088 876 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:07.473580 876 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.459954 31006 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:07.475253 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 90971us
I20260809 07:33:07.471294 31055 pending_rounds.cc:76] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:07.481241 31055 pending_rounds.cc:97] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260809 07:33:07.481526 31055 raft_consensus.cc:2866] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:07.484901 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:07.485044 597 raft_consensus.cc:3032] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:07.485584 597 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:07.486557 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 873) took 3867 ms. Trace:
I20260809 07:33:07.487031 30898 rpcz_store.cc:276] 0809 07:33:03.619227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.672406 (+3053179us) service_pool.cc:227] Handling call
0809 07:33:07.226502 (+554096us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:07.252771 (+ 26269us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.486341 (+233570us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400250dd0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:07.486498 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1273195776}
I20260809 07:33:07.440699 829 consensus_queue.cc:237] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.486603 597 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.492588 31181 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:07.464666 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 898) took 1392 ms. Trace:
I20260809 07:33:07.505718 31006 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:07.481899 876 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.522083 31174 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:07.535907 31048 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:07.536304 1083 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260809 07:33:07.537600 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 896) took 1540 ms. Trace:
W20260809 07:33:07.540483 31005 leader_election.cc:334] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.532s (SENT)
I20260809 07:33:07.505213 32703 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:07.541625 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:07.505889 602 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.537994 30909 rpcz_store.cc:276] 0809 07:33:05.996930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.924543 (+927613us) service_pool.cc:227] Handling call
0809 07:33:06.925275 (+ 732us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:07.429885 (+504610us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.537352 (+107467us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54001b53d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:07.537544 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1067204480}
I20260809 07:33:07.558825 31005 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:07.524626 597 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:07.569962 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:07.573364 807 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:07.573930 807 raft_consensus.cc:686] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:07.576022 655 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260809 07:33:07.574960 807 consensus_queue.cc:237] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.577255 698 raft_consensus.cc:3032] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:07.577639 698 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:07.578084 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 55670.5us
W20260809 07:33:07.580219 31005 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.552s (SENT) [suppressed 46 similar messages]
I20260809 07:33:07.579993 698 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.508104 30891 rpcz_store.cc:276] 0809 07:33:06.072254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:07.072110 (+999856us) service_pool.cc:227] Handling call
0809 07:33:07.172642 (+100532us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:07.173578 (+ 936us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:07.464459 (+290881us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54003c32d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:07.464604 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":228706560}
I20260809 07:33:07.555717 32717 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:07.583453 32717 raft_consensus.cc:686] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.584450 32717 consensus_queue.cc:237] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.581655 698 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.542702 1054 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260809 07:33:07.599761 31006 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:07.602618 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 88614.1us
I20260809 07:33:07.577347 32703 raft_consensus.cc:686] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.577836 1062 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:07.612425 1091 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.106s user 0.000s sys 0.000s
I20260809 07:33:07.564082 1083 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260809 07:33:07.607081 802 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:07.621337 30853 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.639689 32703 consensus_queue.cc:237] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.641880 854 raft_consensus.cc:3032] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:07.642298 854 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:07.642450 1024 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:07.643343 854 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.643952 32199 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 5 LEARNER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:07.643823 1024 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
W20260809 07:33:07.646500 1099 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
W20260809 07:33:07.645643 1089 log.cc:924] Time spent T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.191s user 0.000s sys 0.001s
I20260809 07:33:07.644794 854 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:07.678539 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 59967.9us
W20260809 07:33:07.682782 1104 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.058s user 0.001s sys 0.000s
I20260809 07:33:07.648859 951 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260809 07:33:07.644997 568 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260809 07:33:07.675282 655 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:07.647917 30891 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:07.699010 30891 raft_consensus.cc:595] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
W20260809 07:33:07.710701 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 111747us
I20260809 07:33:07.676875 1024 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:07.714193 1024 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.714859 1007 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:07.715396 1007 raft_consensus.cc:686] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:07.715583 1024 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:07.732833 1007 consensus_queue.cc:237] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.733955 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.675993 802 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.745710 773 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:07.746369 773 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.778589 31046 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:07.741220 664 consensus_queue.cc:1046] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260809 07:33:07.784168 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 55997.7us
I20260809 07:33:07.808516 30891 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
W20260809 07:33:07.959456 1114 log.cc:924] Time spent T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.208s user 0.000s sys 0.007s
I20260809 07:33:07.960194 675 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:07.961426 992 log.cc:924] Time spent T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.145s user 0.005s sys 0.025s
I20260809 07:33:07.962903 30853 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:07.966804 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 212799us
I20260809 07:33:07.967242 802 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.968034 568 consensus_queue.cc:1046] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:07.969581 664 raft_consensus.cc:3032] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
W20260809 07:33:07.966156 31005 leader_election.cc:334] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.763s (SENT)
I20260809 07:33:07.970192 30891 raft_consensus.cc:595] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:07.786715 655 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:07.990476 30891 raft_consensus.cc:603] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:07.989584 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:07.991087 675 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:07.992244 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 21) took 1427 ms. Trace:
I20260809 07:33:08.002388 31181 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:07.992102 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:08.004411 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:07.992668 568 raft_consensus.cc:3032] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:07.989459 664 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:07.981020 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 201422us
I20260809 07:33:07.998586 30853 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.006786 675 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.981705 773 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:07.987167 827 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:08.003371 30744 rpcz_store.cc:276] 0809 07:33:06.593457 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.599637 (+ 6180us) service_pool.cc:227] Handling call
0809 07:33:07.992129 (+1392492us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:07.251326 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:07.251843 (+ 517us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:07.251885 (+ 42us) write_op.cc:435] Acquired schema lock
0809 07:33:07.251908 (+ 23us) tablet.cc:660] Decoding operations
0809 07:33:07.252275 (+ 367us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:07.252362 (+ 87us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:07.252414 (+ 52us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:33:07.279662 (+ 27248us) tablet.cc:699] Row locks acquired
0809 07:33:07.279699 (+ 37us) write_op.cc:260] PREPARE: finished
0809 07:33:07.279860 (+ 161us) write_op.cc:270] Start()
0809 07:33:07.280008 (+ 148us) write_op.cc:276] Timestamp: P: 1786260787279801 usec, L: 0
0809 07:33:07.280040 (+ 32us) op_driver.cc:331] REPLICATION: starting
0809 07:33:07.284542 (+ 4502us) log.cc:842] Serialized 594 byte log entry
0809 07:33:07.542487 (+257945us) op_driver.cc:444] REPLICATION: finished
0809 07:33:07.976976 (+434489us) write_op.cc:301] APPLY: starting
0809 07:33:07.977061 (+ 85us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:07.977164 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:07.977473 (+ 309us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:07.977789 (+ 316us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:07.978046 (+ 257us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:33:07.978106 (+ 60us) write_op.cc:312] APPLY: finished
0809 07:33:07.978382 (+ 276us) log.cc:842] Serialized 22 byte log entry
0809 07:33:07.978563 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:07.978647 (+ 84us) write_op.cc:454] Released schema lock
0809 07:33:07.978703 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:07.979038 (+ 335us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:07.979067 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":411248,"apply.run_cpu_time_us":2202,"apply.run_wall_time_us":2340,"num_ops":1,"prepare.queue_time_us":40448,"prepare.run_cpu_time_us":9464,"prepare.run_wall_time_us":59226,"raft.queue_time_us":70932,"raft.run_cpu_time_us":16023,"raft.run_wall_time_us":252214,"replication_time_us":234058,"thread_start_us":222607,"threads_started":4,"wal-append.queue_time_us":70520}]]}
I20260809 07:33:08.024528 802 consensus_queue.cc:1046] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:08.025254 855 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.018555 32717 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.034861 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.006925 936 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.007539 568 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:08.026832 675 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:08.030720 875 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
I20260809 07:33:08.021767 827 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.026270 802 consensus_queue.cc:1225] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260809 07:33:08.033550 823 raft_consensus.cc:979] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: : Instructing follower a720817fc113446a8ed55e61523b280a to start an election
W20260809 07:33:08.060385 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 274121us
W20260809 07:33:08.066630 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 68176.4us
I20260809 07:33:08.036320 32717 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:08.071200 31059 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:08.071506 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.059651 936 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.072170 31045 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
W20260809 07:33:08.074467 875 raft_consensus.cc:1054] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Unable to promote non-voter 28b9965e21374d26b0ac2014c9bafdff: Illegal state: Leader has not yet committed an operation in its own term
I20260809 07:33:08.069869 32717 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:08.080109 31005 leader_election.cc:334] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.756s (SENT)
I20260809 07:33:08.081859 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.081538 664 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.089586 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.087183 568 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.083624 698 raft_consensus.cc:1067] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Signalling peer a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:08.091082 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.091455 568 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.088300 1054 raft_consensus.cc:3032] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:08.092299 1054 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.092888 875 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:08.093230 951 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:08.093297 875 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.089736 664 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.082273 936 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.097030 603 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.097627 603 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.094762 875 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.099592 603 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.090641 30891 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:08.100346 30891 raft_consensus.cc:595] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.100709 827 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.098006 597 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:08.102205 1137 ts_tablet_manager.cc:927] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Initiating tablet copy from peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
I20260809 07:33:08.093752 951 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.106901 31182 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.100687 30891 raft_consensus.cc:603] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:08.109225 773 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:08.109727 31055 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:36492
I20260809 07:33:08.111455 827 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:08.113570 30853 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.114979 31181 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.115716 802 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.104961 951 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.116354 773 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:08.123175 31055 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:33:08.123607 31055 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:08.131556 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.093255 1054 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.125330 1137 tablet_copy_client.cc:316] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Beginning tablet copy session from remote peer at address 127.28.87.65:33749
I20260809 07:33:08.130566 603 consensus_queue.cc:1046] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260809 07:33:08.114204 875 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.134820 482 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:08.140548 698 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:08.145294 31181 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:08.143733 698 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.146001 698 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:08.150940 30891 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:08.151527 875 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.151607 30891 raft_consensus.cc:595] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.155258 30891 raft_consensus.cc:603] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:08.156786 1083 raft_consensus.cc:3032] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:08.157181 1083 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.158128 32717 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.168042 31051 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:08.158018 1083 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.162859 603 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:08.174736 1083 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.173938 32717 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.177048 854 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:08.177867 603 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:08.178283 603 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.178596 1054 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.179147 603 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.185616 32717 consensus_queue.cc:1046] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:08.183158 31048 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:08.186249 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.186913 31051 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:08.185787 603 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.188906 1050 raft_consensus.cc:3032] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:08.189272 1050 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.190069 1050 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.191282 1050 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:33:08.187264 1132 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.081s user 0.000s sys 0.001s
I20260809 07:33:08.195222 30891 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:08.195828 30891 raft_consensus.cc:595] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.196177 30891 raft_consensus.cc:603] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:08.196982 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.197500 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:33:08.197924 875 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:08.198386 875 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.198796 875 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:33:08.195968 1140 log.cc:924] Time spent T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.090s user 0.004s sys 0.000s
I20260809 07:33:08.201951 31181 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:08.201900 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:08.207336 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:08.165887 698 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.211823 1014 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:08.212743 875 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.213048 664 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.213739 664 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.215709 664 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.172117 951 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.219110 951 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.221071 698 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.223479 30852 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.224684 603 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.225119 603 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:08.225530 603 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.226001 30853 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.227954 30853 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.229234 30853 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.229985 482 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.230592 482 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.231159 698 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 07:33:08.232398 31005 leader_election.cc:334] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.598s (SENT)
I20260809 07:33:08.234424 698 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.237008 698 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.217737 875 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.223518 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.241183 936 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.241537 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.242105 875 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:08.242753 875 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.243207 875 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.244072 30742 catalog_manager.cc:5495] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:08.245318 31174 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:08.246680 664 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:08.246708 1014 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:08.247196 664 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.247566 664 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.247555 603 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.242085 698 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.254273 31181 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:08.256050 664 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.257143 1014 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:08.262666 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.263314 31182 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.263785 664 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.255061 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:08.261966 31005 leader_election.cc:334] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.900s (SENT)
I20260809 07:33:08.252197 603 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.267138 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.183815 31055 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.274093 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:08.292641 1014 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:08.298548 30853 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.300141 827 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
W20260809 07:33:08.300595 827 raft_consensus.cc:1054] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Unable to promote non-voter 28b9965e21374d26b0ac2014c9bafdff: Illegal state: Leader has not yet committed an operation in its own term
I20260809 07:33:08.287147 664 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:08.304390 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.305740 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.269838 875 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.307328 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.306663 31046 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:08.302937 30742 catalog_manager.cc:5495] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:08.290879 603 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:08.307905 31051 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.307951 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.309110 31049 raft_consensus.cc:2354] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
I20260809 07:33:08.310076 30742 catalog_manager.cc:5495] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:08.312587 30742 catalog_manager.cc:5495] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:08.314603 30742 catalog_manager.cc:5495] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:08.315922 31048 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:08.316766 31051 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:08.319410 30852 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.319032 30742 catalog_manager.cc:5495] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:08.322798 30852 leader_election.cc:398] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
I20260809 07:33:08.325163 30852 leader_election.cc:401] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260809 07:33:08.320721 823 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:08.334714 31055 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.338945 30742 catalog_manager.cc:5495] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 3, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:08.339833 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.341053 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.341315 951 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.342135 597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:34562 (request call id 0) took 9314 ms. Trace:
I20260809 07:33:08.342563 597 rpcz_store.cc:276] 0809 07:32:59.027443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:32:59.173854 (+146411us) service_pool.cc:227] Handling call
0809 07:32:59.330441 (+156587us) spinlock_profiling.cc:243] Waited 61.5 ms on lock 0x7b540011cc20. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094e8edc0 00007f4094effbd2 00007f4094efe890 00007f4099e3bf39 00007f4093527704 00007f4093527692 00007f40935275c1 00007f409352753c 00007f40935267f2 00007f409332394c
0809 07:32:59.431964 (+101523us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b2c00012038. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ebada 00007f40938eb763 00007f40938eb701 00007f4094f34b12 00007f4094f348ed 00007f4094efff84 00007f4094eff743 00007f4099e3bf39 00007f4093527704 00007f4093527692
0809 07:32:59.500573 (+ 68609us) log.cc:842] Serialized 499 byte log entry
0809 07:32:59.930185 (+429612us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b1800229200. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094f35f11 00007f4094e8af91 00007f4094e73545 00007f4094e78771 00007f4094e786f9 00007f4094e78689 00007f4094e78651 00007f4094e7781d 00007f409b235204 00007f409b235039
0809 07:32:59.936402 (+ 6217us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b240018b048. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091e29cf4 00007f4091e27dfb 00007f4091eb6559 00007f4091eb61e5 00007f4091eb582d 00007f4094b6446e 00007f4094b7bc0c 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551
0809 07:33:00.270612 (+334210us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b1800370a80. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094e8aa4f 00007f4094e73545 00007f4094e78771
0809 07:33:00.305911 (+ 35299us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b1800370a80. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094e92416 00007f4094e8bdaa 00007f4094e8b014
0809 07:33:00.306758 (+ 847us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b540011cc20. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094e92416 00007f4094e8bdaa 00007f4094e8b014 00007f4094e73545 00007f4094e78771 00007f4094e786f9 00007f4094e78689 00007f4094e78651 00007f4094e7781d 00007f409b235204
0809 07:33:00.598812 (+292054us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b540011cc20. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094e8aa4f 00007f4094e73545 00007f4094e78771 00007f4094e786f9 00007f4094e78689 00007f4094e78651 00007f4094e7781d 00007f409b235204 00007f409b235039 00007f4091fd16b6
0809 07:33:08.342006 (+7743194us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":133,"mutex_wait_us":35294,"raft.queue_time_us":3487,"raft.run_cpu_time_us":15805,"raft.run_wall_time_us":1590717,"spinlock_wait_cycles":192482176,"thread_start_us":226930,"threads_started":1,"wal-append.queue_time_us":339080,"wal-append.run_cpu_time_us":6100,"wal-append.run_wall_time_us":1181162}
I20260809 07:33:08.345201 1083 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:08.345345 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.346114 655 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:08.346451 30853 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.346649 655 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.347729 655 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.349393 482 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:08.322383 1054 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 07:33:08.350482 1054 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.350898 1054 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.353406 802 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.321894 664 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.354457 664 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.335114 936 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.354324 802 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.340445 30742 catalog_manager.cc:5495] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 4 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:08.346267 30852 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.362126 664 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.363063 698 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:08.365092 698 raft_consensus.cc:686] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.365474 31182 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.365922 698 consensus_queue.cc:237] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.371160 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.373066 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.378346 32717 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:08.378834 32717 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.380082 31182 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:08.382826 951 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:08.379768 30742 catalog_manager.cc:5495] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 4, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 4 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:08.387552 875 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.387310 30742 catalog_manager.cc:5495] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:08.389024 30742 catalog_manager.cc:5495] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 2, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 2 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
W20260809 07:33:08.387621 30827 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Replica 2c9de56c3fa3487c9dac990675c34be5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:08.391005 30742 catalog_manager.cc:5495] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:08.383046 32717 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.393957 30742 catalog_manager.cc:5495] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:08.405864 31182 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:08.410107 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.412411 31177 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
W20260809 07:33:08.422636 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 81667.9us
I20260809 07:33:08.423491 1034 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:08.424091 1034 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.425429 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:08.426610 603 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.427156 603 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.427392 1049 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.428301 699 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:08.428704 31165 raft_consensus.cc:1261] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.429126 603 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.430616 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.431346 31055 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.431886 31177 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:08.439193 802 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.425046 1034 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.442236 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.443128 31055 raft_consensus.cc:2396] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:08.424242 664 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:08.451048 32717 consensus_queue.cc:1046] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:08.452396 32717 raft_consensus.cc:3032] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:08.452746 32717 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.454137 31177 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:08.454921 30742 catalog_manager.cc:5495] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:08.457508 31174 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:08.458979 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.466776 32717 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.468102 31170 raft_consensus.cc:1261] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:33:08.468029 32717 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.473448 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.468461 664 consensus_queue.cc:1046] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260809 07:33:08.476818 1049 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:08.477676 699 raft_consensus.cc:3032] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:08.478056 699 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.478968 699 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.480537 30930 tablet_copy_service.cc:140] P 2c9de56c3fa3487c9dac990675c34be5: Received BeginTabletCopySession request for tablet 335fccb907184174b4a6428ce262e7dc from peer 28b9965e21374d26b0ac2014c9bafdff ({username='slave'} at 127.0.0.1:54234)
I20260809 07:33:08.491983 30930 tablet_copy_service.cc:161] P 2c9de56c3fa3487c9dac990675c34be5: Beginning new tablet copy session on tablet 335fccb907184174b4a6428ce262e7dc from peer 28b9965e21374d26b0ac2014c9bafdff at {username='slave'} at 127.0.0.1:54234: session id = 28b9965e21374d26b0ac2014c9bafdff-335fccb907184174b4a6428ce262e7dc
I20260809 07:33:08.492645 875 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.499141 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.500005 664 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.500427 664 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.500733 664 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.503239 30930 tablet_copy_source_session.cc:205] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:33:08.505968 664 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.507301 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.507401 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:08.508926 664 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.509423 655 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.506817 1137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fccb907184174b4a6428ce262e7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:33:08.493852 699 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.511143 655 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.511474 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.505993 951 raft_consensus.cc:3032] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:08.512277 951 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:08.510692 31174 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:08.475253 1083 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
I20260809 07:33:08.513129 951 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.514101 655 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.510632 31170 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:08.515931 31177 raft_consensus.cc:1203] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[4.2-4.2] Dedup: 4.2->[]
I20260809 07:33:08.516459 951 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.516999 31177 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260809 07:33:08.519466 31182 raft_consensus.cc:1203] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-3.3] Dedup: 3.3->[]
I20260809 07:33:08.510012 1049 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.520071 31182 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:08.519812 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.521149 31055 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:08.522415 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.500599 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.523571 664 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.523991 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.524643 664 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.524993 664 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.525342 664 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.477171 1054 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.531890 1054 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.542832 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.542869 1083 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:08.548938 1083 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:08.549144 1050 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.549356 1083 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:08.549721 1050 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.552726 1050 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.553553 875 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:08.554041 875 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:08.554419 875 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.554811 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.555634 31165 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.556473 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.557062 31165 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:08.557397 31170 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:08.558130 31174 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.558883 31172 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:08.560732 31181 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:08.557639 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.553555 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.564214 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.564999 807 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 07:33:08.565392 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.536073 664 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.566147 807 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.577512 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.578034 31181 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.567041 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.570381 31169 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:08.572154 1049 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:08.569586 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.583683 1049 raft_consensus.cc:3032] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:08.584036 1049 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.567834 875 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.549005 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.584832 1049 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.568756 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.578295 31174 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:08.577524 655 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.579850 698 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.572117 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.580627 31048 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:08.586570 655 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.580826 802 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.586839 698 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.587666 802 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.588598 655 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.589043 698 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.581560 603 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.586189 1049 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:08.585336 31055 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
W20260809 07:33:08.577009 31005 leader_election.cc:334] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 2.165s (SENT)
I20260809 07:33:08.571352 1050 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:08.589552 802 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.591528 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.592098 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.592392 31048 raft_consensus.cc:2396] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:08.592270 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.591636 807 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.592917 31051 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:08.592866 31049 raft_consensus.cc:2379] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.595439 31172 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
W20260809 07:33:08.593251 31005 leader_election.cc:334] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.672s (SENT)
I20260809 07:33:08.589951 603 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.598363 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.598785 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.598975 603 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.600142 31177 raft_consensus.cc:2396] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:08.600595 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.579105 664 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:08.602152 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.599143 30891 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.603452 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.603837 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.601982 1034 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260809 07:33:08.605558 1034 raft_consensus.cc:3032] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:08.605873 1034 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.606621 30909 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:08.605927 875 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:08.606546 31005 leader_election.cc:334] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 2.194s (SENT)
I20260809 07:33:08.607529 807 consensus_queue.cc:1046] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:08.608728 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.609279 31172 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.610561 568 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:08.613396 31182 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:08.614653 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.607007 31177 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:08.608585 807 raft_consensus.cc:3032] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:08.614410 827 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260809 07:33:08.623377 603 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:08.623751 603 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.624102 603 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.606856 1034 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.624383 699 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:08.625622 1007 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:08.626071 1007 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.622607 31169 raft_consensus.cc:2354] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
I20260809 07:33:08.625468 1034 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.619999 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.630591 31049 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:08.631361 31055 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:08.631960 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.600122 1050 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.633497 31172 raft_consensus.cc:2396] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:08.635114 30852 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.634614 655 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.596411 802 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260809 07:33:08.637693 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.627789 807 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:08.636210 664 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260809 07:33:08.642906 1083 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.637169 655 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:08.645416 655 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.643091 802 consensus_queue.cc:1225] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:08.636431 1054 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:08.627002 1007 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.646525 1083 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.652861 30906 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:08.634989 568 consensus_queue.cc:1046] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260809 07:33:08.662588 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.663142 31055 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:08.664414 807 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.635136 1050 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:08.667666 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 53610.2us
I20260809 07:33:08.667838 31172 raft_consensus.cc:1226] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:08.669360 1083 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.676983 699 consensus_queue.cc:1046] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:08.677749 1083 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.678229 32717 raft_consensus.cc:3032] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:08.678622 32717 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:08.678983 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.679651 30904 raft_consensus.cc:2354] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:08.679363 32717 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.679843 1083 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.661568 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.680658 32717 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.680922 30891 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:08.681459 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.682049 31049 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:08.669548 1054 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.677052 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.683526 30900 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.673436 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.684455 30906 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.678052 827 raft_consensus.cc:3032] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:08.683660 30852 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.674742 1007 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.685377 827 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:08.686403 1054 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.690651 807 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.691346 827 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.688638 698 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:08.692734 698 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:08.695159 698 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.698526 827 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:08.685204 936 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:08.700338 936 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.701148 936 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.702236 936 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:08.703691 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.704674 30891 raft_consensus.cc:2379] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 2: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.705169 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.705670 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.706127 30890 raft_consensus.cc:2396] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:08.702569 698 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.707744 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.708120 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.707979 1050 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.709368 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.710155 31045 raft_consensus.cc:2396] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:08.708945 31048 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260809 07:33:08.711720 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.711957 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.712141 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.706282 30904 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.709386 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.689640 568 consensus_queue.cc:1046] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260809 07:33:08.714133 568 raft_consensus.cc:3032] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 07:33:08.714476 568 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:08.715735 698 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.713199 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.716522 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.710101 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.718477 31046 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.718835 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.719378 30904 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.716040 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.720346 31048 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.720721 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.721315 30890 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.724574 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.725243 30896 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.727332 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.726691 30911 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.728981 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.732304 31046 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:08.715322 568 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.741416 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.742166 31049 raft_consensus.cc:2379] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.763702 31059 raft_consensus.cc:1261] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:08.747097 1083 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.769034 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.746840 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.764290 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.773216 31046 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.733870 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.776160 31059 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:08.738456 603 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.779232 30909 raft_consensus.cc:2379] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.780028 603 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.781363 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.782025 30909 raft_consensus.cc:2354] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6.
I20260809 07:33:08.773139 31057 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.782874 30900 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.783164 31055 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:08.773799 30906 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.786211 31172 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:08.776319 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.792176 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.792797 30911 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.793033 30900 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.793619 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.791676 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.794982 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.795035 31059 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.797591 31057 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:08.798136 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.802170 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.803463 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.801187 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.804848 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.805260 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.801330 30906 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.801211 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.808225 30911 raft_consensus.cc:2379] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.787688 1083 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.813032 31058 raft_consensus.cc:1261] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.814556 31054 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:08.819727 30911 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.820732 30898 raft_consensus.cc:1261] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:08.808007 698 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.821698 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.810856 802 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:08.822424 30906 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:08.821583 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.819873 31048 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:08.823889 31055 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:08.824663 30852 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.825066 31058 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:08.826050 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.815960 568 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:08.823791 30909 raft_consensus.cc:2362] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4. Re-sending same reply.
I20260809 07:33:08.831714 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.840839 31059 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:08.842609 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.843910 30909 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.830354 597 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:08.850720 30900 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:08.852468 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.852533 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.857568 30904 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
W20260809 07:33:08.859766 31005 leader_election.cc:334] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 2.170s (SENT)
I20260809 07:33:08.853389 30906 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:08.860196 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:08.854802 30900 raft_consensus.cc:2362] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5. Re-sending same reply.
I20260809 07:33:08.851521 31196 tablet_copy_service.cc:342] P 12107e38eb6d424abc4f45f16a9a1560: Request end of tablet copy session 28b9965e21374d26b0ac2014c9bafdff-3bbfdffc27ac46a0a18582f85cd4a00a received from {username='slave'} at 127.0.0.1:45380
I20260809 07:33:08.858678 597 raft_consensus.cc:686] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.863027 31196 tablet_copy_service.cc:433] P 12107e38eb6d424abc4f45f16a9a1560: ending tablet copy session 28b9965e21374d26b0ac2014c9bafdff-3bbfdffc27ac46a0a18582f85cd4a00a on tablet 3bbfdffc27ac46a0a18582f85cd4a00a with peer 28b9965e21374d26b0ac2014c9bafdff
I20260809 07:33:08.863405 31120 leader_election.cc:398] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6.
I20260809 07:33:08.863595 597 consensus_queue.cc:237] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.864619 1083 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.830242 802 raft_consensus.cc:686] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:08.865583 802 consensus_queue.cc:237] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.867029 823 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.867480 823 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.869376 823 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.872067 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.872684 31058 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:08.874177 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.875808 31120 leader_election.cc:401] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260809 07:33:08.876332 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.876983 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.879878 31059 raft_consensus.cc:1261] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:08.880549 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.877557 30909 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:08.881754 1007 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:08.882025 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.883200 602 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:08.883612 1171 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:08.884378 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.885246 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.886376 807 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.887610 807 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
I20260809 07:33:08.888058 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.883070 854 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:08.883131 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.888712 854 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:08.889045 664 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:08.890172 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:08.890993 1085 log.cc:924] Time spent T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Append to log took a long time: real 0.077s user 0.001s sys 0.000s
I20260809 07:33:08.887480 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.892405 31046 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
W20260809 07:33:08.892704 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 60497.5us
I20260809 07:33:08.893365 664 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260809 07:33:08.894866 1173 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260809 07:33:08.887729 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.895241 655 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.895687 31048 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:08.853716 1054 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:08.897154 1054 raft_consensus.cc:686] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:08.888015 807 raft_consensus.cc:1054] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Unable to promote non-voter 28b9965e21374d26b0ac2014c9bafdff: Illegal state: Leader has not yet committed an operation in its own term
I20260809 07:33:08.888155 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.897943 875 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:08.899803 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.900916 1049 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.901383 1049 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.877905 30890 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:08.903013 31048 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:08.905002 30899 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:08.905627 31058 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:08.906241 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.897431 31051 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:08.905812 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.907250 30899 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.909780 31054 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:08.912374 31045 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:08.918728 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.922755 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.922941 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.923507 31059 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:08.897302 655 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.907330 30896 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.924342 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.925098 30900 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.925822 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.921253 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.926513 30900 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.826709 698 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:08.919255 30891 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:08.869385 1137 tablet_copy_client.cc:791] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Starting download of 0 data blocks...
I20260809 07:33:08.924026 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.929193 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.942977 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.942807 30904 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.943835 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.944994 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.945484 655 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:08.927733 30906 raft_consensus.cc:2379] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 3: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.943332 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.889098 854 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:08.947206 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.948468 30891 raft_consensus.cc:2362] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5. Re-sending same reply.
I20260809 07:33:08.949200 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.949743 30891 raft_consensus.cc:2362] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5. Re-sending same reply.
I20260809 07:33:08.947640 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:08.950994 31058 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:08.951691 30904 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:08.952857 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.952773 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.953644 30904 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:08.933879 31051 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:08.935906 698 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.936316 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:08.936534 875 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:08.936436 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.971537 30900 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:08.971467 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.937286 807 consensus_queue.cc:1046] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:08.936460 1171 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6.
I20260809 07:33:08.941138 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.978207 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.903548 1049 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:08.949229 30899 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:08.960000 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.979826 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.981755 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:08.982769 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46540.2us
I20260809 07:33:08.952422 30898 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:08.968703 30852 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:08.985191 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.979182 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.986815 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.987412 30904 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:08.987617 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.972136 30909 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:08.971010 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.942842 1137 tablet_copy_client.cc:655] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Starting download of 1 WAL segments...
I20260809 07:33:08.974637 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.994416 31172 raft_consensus.cc:2396] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 07:33:08.989560 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:09.001029 30906 raft_consensus.cc:2362] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3. Re-sending same reply.
I20260809 07:33:08.903662 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.975596 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:09.007038 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:09.006963 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:08.985160 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.987964 31174 raft_consensus.cc:1261] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:09.011000 31174 pending_rounds.cc:76] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 2
I20260809 07:33:09.011251 31174 pending_rounds.cc:97] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260809 07:33:09.011478 31174 raft_consensus.cc:2866] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:08.954183 30891 raft_consensus.cc:2362] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5. Re-sending same reply.
I20260809 07:33:09.013015 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.988504 30899 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:08.898049 1054 consensus_queue.cc:237] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:08.946031 827 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260809 07:33:08.990994 31177 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:08.992460 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:08.988314 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:09.022219 30904 raft_consensus.cc:2362] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5. Re-sending same reply.
I20260809 07:33:08.997014 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.997920 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:08.998294 1049 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:08.998754 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:08.978286 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.973016 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:08.988644 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:09.025274 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:09.026180 30906 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:09.027513 31005 leader_election.cc:398] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:09.027829 31005 leader_election.cc:401] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260809 07:33:08.993796 698 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:09.032006 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:09.034803 30904 raft_consensus.cc:2362] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6. Re-sending same reply.
I20260809 07:33:09.000614 1171 raft_consensus.cc:3032] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:09.030841 30898 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:09.036623 30898 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:09.036844 664 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:09.038026 30898 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:09.039458 699 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:09.019587 1034 consensus_queue.cc:1046] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260809 07:33:09.019315 951 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:09.021541 655 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260809 07:33:09.031001 1049 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.042665 875 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:09.043324 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:09.088943 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 45710.5us
I20260809 07:33:09.044140 30853 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.034412 698 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:09.046789 936 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 07:33:09.047432 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:09.091459 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:09.053736 30900 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:09.081844 30904 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:09.082894 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:09.083531 30909 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:09.092517 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:09.080742 30891 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:09.060084 30890 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:09.079540 30899 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:09.103101 30909 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:09.103543 30898 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260809 07:33:09.088105 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 94029.4us
I20260809 07:33:09.044521 1171 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:09.072338 31049 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:09.093781 30900 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260809 07:33:09.111219 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 67147.2us
I20260809 07:33:09.108050 30899 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:09.111974 30853 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.110864 1171 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.095253 698 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:09.120157 31175 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:09.090545 1007 raft_consensus.cc:3032] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
W20260809 07:33:09.120741 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 73391.2us
I20260809 07:33:09.098532 807 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:09.102530 602 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:09.132263 30900 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:09.134083 30853 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.093365 951 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:09.102696 823 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:09.133205 30899 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260809 07:33:09.109738 1034 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:33:09.135898 31005 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:09.132638 30909 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 2. Current term is 3. Ops: []
I20260809 07:33:09.108711 568 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:09.130793 1171 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:09.139101 802 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:09.157249 30904 raft_consensus.cc:1226] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:09.161947 30853 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.131585 1007 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:09.166128 30909 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:09.136829 951 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:09.171334 30909 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:09.140641 823 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.180845 30853 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.166116 655 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:09.167856 699 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:09.170482 951 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:09.171958 807 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:09.198987 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 63176.6us
I20260809 07:33:09.180967 30890 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:09.210741 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.190471 597 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:09.178781 936 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:33:09.187343 602 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.040s
W20260809 07:33:09.238011 1179 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.100s user 0.000s sys 0.001s
I20260809 07:33:09.238370 602 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 07:33:09.239923 1180 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260809 07:33:09.240257 602 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:09.218181 1083 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260809 07:33:09.244390 1137 tablet_copy_client.cc:531] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:33:09.247540 30899 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
W20260809 07:33:09.249796 1177 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.119s user 0.000s sys 0.001s
I20260809 07:33:09.261143 1195 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260809 07:33:09.275017 31174 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260809 07:33:09.279685 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 76311.2us
I20260809 07:33:09.283794 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:09.410194 31174 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:09.413555 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:09.427233 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:09.284843 1049 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:09.217554 1007 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.231158 1183 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.193s
I20260809 07:33:09.423491 1189 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:09.284674 1050 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.154s
W20260809 07:33:09.470808 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 336201us
I20260809 07:33:09.475047 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.468713 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:09.443657 1189 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:09.443411 568 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:09.486402 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.475557 699 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:09.497303 32717 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:09.428784 807 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.478456 1137 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Bootstrap starting.
I20260809 07:33:09.598371 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:09.488582 875 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 07:33:09.512096 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:09.498358 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:09.487740 823 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:09.490156 1189 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:09.613059 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 402375us
I20260809 07:33:09.545572 1201 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:09.527473 1193 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260809 07:33:09.631402 802 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:09.639124 1204 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20260809 07:33:09.645520 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43208
I20260809 07:33:09.646588 31057 raft_consensus.cc:1261] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260809 07:33:09.660609 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 162386us
W20260809 07:33:09.536681 854 consensus_meta.cc:287] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.561s user 0.009s sys 0.000s
I20260809 07:33:09.663475 31058 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:09.662166 1050 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
W20260809 07:33:09.537956 1176 log.cc:924] Time spent T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.347s user 0.000s sys 0.001s
I20260809 07:33:09.513568 32717 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:09.495190 699 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:09.666469 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 171378us
I20260809 07:33:09.683727 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:09.684163 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:09.658686 698 consensus_meta.cc:287] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.520s user 0.008s sys 0.001s
I20260809 07:33:09.676735 30853 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.665637 875 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:09.700866 32717 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.541282 1225 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.143s
I20260809 07:33:09.613574 568 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.683151 854 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:09.758608 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 78491.6us
W20260809 07:33:09.796797 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 118122us
I20260809 07:33:09.797426 30900 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:09.682260 30890 raft_consensus.cc:595] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Received request to transfer leadership
W20260809 07:33:09.759289 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 48033.1us
I20260809 07:33:09.685868 699 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:09.890988 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 214374us
W20260809 07:33:09.861166 1171 consensus_meta.cc:287] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.700s user 0.000s sys 0.023s
I20260809 07:33:09.703967 875 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.918146 827 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:09.918428 936 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 07:33:09.821183 1206 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.275s user 0.001s sys 0.000s
W20260809 07:33:09.702585 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 287984us
I20260809 07:33:09.908365 30900 raft_consensus.cc:595] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.934791 30900 raft_consensus.cc:603] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260809 07:33:09.761196 1191 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.323s user 0.002s sys 0.002s
W20260809 07:33:09.817278 1207 log.cc:924] Time spent T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.401s user 0.003s sys 0.002s
I20260809 07:33:09.917577 30852 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:09.891999 1193 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
I20260809 07:33:09.806195 30744 catalog_manager.cc:5495] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
W20260809 07:33:09.938861 1193 raft_consensus.cc:1054] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Unable to promote non-voter 28b9965e21374d26b0ac2014c9bafdff: Illegal state: Leader has not yet committed an operation in its own term
W20260809 07:33:09.860766 951 consensus_meta.cc:287] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.650s user 0.013s sys 0.004s
I20260809 07:33:09.792928 1197 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
I20260809 07:33:09.700158 1050 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:09.924685 827 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:09.925043 30896 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:09.924865 875 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:09.919677 1199 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:33:09.941704 664 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:09.956398 664 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:09.826613 1203 log.cc:924] Time spent T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.279s user 0.001s sys 0.003s
I20260809 07:33:09.920305 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:09.798496 1222 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:09.702956 1049 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:09.711931 1211 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.249s
I20260809 07:33:09.925302 1195 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:33:09.931965 854 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:09.694144 698 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:09.759040 1054 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260809 07:33:09.857474 1034 consensus_meta.cc:287] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.712s user 0.014s sys 0.007s
I20260809 07:33:09.983996 31046 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:09.922245 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:09.712896 1201 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:09.789860 1007 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:09.949862 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 760) took 1011 ms. Trace:
I20260809 07:33:09.950799 31175 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:09.951719 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:09.954602 1224 consensus_queue.cc:1046] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260809 07:33:09.959620 31058 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:09.959811 664 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:09.960609 827 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:09.961707 1050 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.965286 1189 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.966847 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:09.968056 1171 consensus_queue.cc:1046] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 07:33:09.971101 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 50865.4us
I20260809 07:33:09.993718 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:09.958601 1197 raft_consensus.cc:1054] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Unable to promote non-voter 28b9965e21374d26b0ac2014c9bafdff: Illegal state: Leader has not yet committed an operation in its own term
W20260809 07:33:09.980419 30827 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 87ac258020ac4e29b08523647d3ab361, TS 12107e38eb6d424abc4f45f16a9a1560 -> TS 28b9965e21374d26b0ac2014c9bafdff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.975005 602 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 07:33:09.985023 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 62864.8us
I20260809 07:33:09.988541 30896 rpcz_store.cc:276] 0809 07:33:08.938405 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:08.970739 (+ 32334us) service_pool.cc:227] Handling call
0809 07:33:09.949692 (+978953us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54001a4ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:09.949811 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2081710464}
I20260809 07:33:10.001322 30853 leader_election.cc:398] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
I20260809 07:33:09.990221 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:09.943171 951 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.988175 1183 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:10.002856 1183 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:10.003247 1183 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.003819 951 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:10.004307 1049 raft_consensus.cc:686] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:10.005199 1049 consensus_queue.cc:237] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:09.988344 1034 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:33:10.012346 30853 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:09.996508 30827 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 87ac258020ac4e29b08523647d3ab361, TS 12107e38eb6d424abc4f45f16a9a1560 -> TS 28b9965e21374d26b0ac2014c9bafdff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.002543 31059 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.018515 1231 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260809 07:33:10.019654 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 52877.7us
I20260809 07:33:09.997151 30890 raft_consensus.cc:595] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.020375 1171 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:10.028365 1225 raft_consensus.cc:3032] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 07:33:10.030813 655 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:10.032339 1209 raft_consensus.cc:979] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: : Instructing follower a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:09.992197 1054 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:09.990929 827 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.043692 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:10.014209 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:10.018664 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 936) took 1134 ms. Trace:
I20260809 07:33:10.047127 30853 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.011474 1226 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:10.010514 1183 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.050879 30742 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40566 (request call id 151) took 3512 ms. Trace:
I20260809 07:33:09.976805 1222 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.072335 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:10.074007 698 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.069943 854 raft_consensus.cc:1067] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Signalling peer a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:10.074632 1222 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.048524 30890 raft_consensus.cc:603] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:10.056624 1034 consensus_queue.cc:1046] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260809 07:33:10.044263 1231 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:10.077178 1231 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.077379 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:10.078063 31046 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:10.074816 1224 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:10.057286 1230 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260809 07:33:10.078672 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:10.076995 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:10.079281 30890 raft_consensus.cc:595] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.079591 31055 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:10.052131 1225 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.048933 1083 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:10.079634 30890 raft_consensus.cc:603] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:10.081701 1201 raft_consensus.cc:3032] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:10.082068 1201 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.083076 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:10.082831 1201 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.083668 30890 raft_consensus.cc:595] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.075285 1054 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.084051 1201 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.084034 30890 raft_consensus.cc:603] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:10.086963 1049 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:10.087533 1049 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:10.084888 1225 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.088774 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.089710 1225 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:10.088315 1049 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.089066 1034 consensus_queue.cc:1225] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260809 07:33:10.091162 31055 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:10.091711 1183 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.064772 1171 raft_consensus.cc:686] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:10.087051 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 87630us
I20260809 07:33:10.074299 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.094702 31051 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:36492
W20260809 07:33:10.097376 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 50327.6us
I20260809 07:33:10.097816 31051 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:33:10.097275 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:10.098207 31051 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.098685 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.065565 30891 rpcz_store.cc:276] 0809 07:33:08.884008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:09.102722 (+218714us) service_pool.cc:227] Handling call
0809 07:33:09.103109 (+ 387us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:09.103467 (+ 358us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:09.103496 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:09.137347 (+ 33851us) log.cc:842] Serialized 52 byte log entry
0809 07:33:09.428593 (+291246us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:09.469632 (+ 41039us) log.cc:842] Serialized 14 byte log entry
0809 07:33:09.491997 (+ 22365us) log.cc:842] Serialized 14 byte log entry
0809 07:33:09.494347 (+ 2350us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:09.494428 (+ 81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:09.536337 (+ 41909us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b1800308ce0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:09.826064 (+289727us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:10.018407 (+192343us) raft_consensus.cc:1671] finished
0809 07:33:10.018452 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:10.018598 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33400,"prepare.queue_time_us":109673,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":133610,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":8004,"spinlock_wait_cycles":10160768,"thread_start_us":259124,"threads_started":1,"wal-append.queue_time_us":340345}
I20260809 07:33:10.095381 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:10.093770 30852 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:10.101843 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.100176 31046 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:10.103098 31059 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:10.103950 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:10.071941 30742 rpcz_store.cc:276] 0809 07:33:06.538795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:06.543738 (+ 4943us) service_pool.cc:227] Handling call
0809 07:33:10.050784 (+3507046us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:08.595612 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:08.598210 (+ 2598us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:08.598251 (+ 41us) write_op.cc:435] Acquired schema lock
0809 07:33:08.598267 (+ 16us) tablet.cc:660] Decoding operations
0809 07:33:08.641942 (+ 43675us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:08.651751 (+ 9809us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:08.653132 (+ 1381us) tablet.cc:683] Acquiring locks for 14 operations
0809 07:33:08.797086 (+143954us) tablet.cc:699] Row locks acquired
0809 07:33:08.797103 (+ 17us) write_op.cc:260] PREPARE: finished
0809 07:33:08.797275 (+ 172us) write_op.cc:270] Start()
0809 07:33:08.797391 (+ 116us) write_op.cc:276] Timestamp: P: 1786260788797204 usec, L: 0
0809 07:33:08.797406 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 07:33:08.802084 (+ 4678us) log.cc:842] Serialized 5543 byte log entry
0809 07:33:09.033156 (+231072us) op_driver.cc:444] REPLICATION: finished
0809 07:33:09.443500 (+410344us) write_op.cc:301] APPLY: starting
0809 07:33:09.459632 (+ 16132us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:09.574385 (+114753us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:09.660977 (+ 86592us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:09.753423 (+ 92446us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:09.753665 (+ 242us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0809 07:33:09.753713 (+ 48us) write_op.cc:312] APPLY: finished
0809 07:33:09.754033 (+ 320us) log.cc:842] Serialized 100 byte log entry
0809 07:33:09.754208 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:09.754429 (+ 221us) write_op.cc:454] Released schema lock
0809 07:33:09.754479 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:09.754746 (+ 267us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:09.754779 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12445,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":384307,"apply.run_cpu_time_us":5256,"apply.run_wall_time_us":317937,"num_ops":14,"prepare.queue_time_us":62174,"prepare.run_cpu_time_us":6522,"prepare.run_wall_time_us":206763,"replication_time_us":230866,"thread_start_us":216880,"threads_started":2}]]}
I20260809 07:33:10.106068 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:10.106541 30852 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:10.086077 1054 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.110109 31051 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:10.113745 30853 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.118033 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:10.130952 31045 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:10.091964 1246 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:10.134912 31051 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.097111 664 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:10.119726 30852 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:10.136770 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.137367 31175 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.138708 30853 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.139348 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:10.141937 31182 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:10.102737 1224 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.102806 1083 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:10.102926 1209 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:10.147269 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:10.147892 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.112812 1199 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:10.112742 1230 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:10.112605 854 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:10.138062 1183 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:10.112880 1034 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:10.134186 1222 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:10.130645 807 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.153348 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:10.111466 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:10.138202 1226 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:10.080081 827 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.138160 951 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:10.154651 854 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:10.155004 31120 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.139906 1246 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:10.166957 1240 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.122s user 0.000s sys 0.001s
I20260809 07:33:10.155905 655 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:10.145798 1054 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260809 07:33:10.145586 698 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:10.169487 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:10.143842 1193 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:10.201089 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47836.5us
I20260809 07:33:10.166074 1213 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.220814 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:10.221374 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 74190.2us
I20260809 07:33:10.209479 1014 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:10.252555 31170 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:10.216495 597 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:10.259457 30891 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:10.216856 568 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:10.219149 854 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.219313 30890 raft_consensus.cc:595] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:10.218770 1222 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:10.263274 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.262143 1193 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.219035 1189 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.274677 30890 raft_consensus.cc:603] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:10.251700 655 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:10.275753 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.278250 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.279021 31170 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:10.255165 827 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.279563 1214 consensus_queue.cc:1046] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260809 07:33:10.252141 1034 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:10.288388 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:33:10.290146 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:10.263877 664 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.295161 1193 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:10.298414 699 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.311074 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:10.277953 1183 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:10.282460 597 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:10.259784 602 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260809 07:33:10.283954 655 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.331514 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 938) took 1421 ms. Trace:
I20260809 07:33:10.210999 1171 consensus_queue.cc:237] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.335175 31005 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:10.347499 30898 rpcz_store.cc:276] 0809 07:33:08.884543 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:09.104228 (+219685us) service_pool.cc:227] Handling call
0809 07:33:09.122714 (+ 18486us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:09.173944 (+ 51230us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:09.173977 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:09.174598 (+ 621us) log.cc:842] Serialized 26 byte log entry
0809 07:33:09.280295 (+105697us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:09.280329 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:09.281434 (+ 1105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:09.820594 (+539160us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:10.260872 (+440278us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b240018b048. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091e29cf4 00007f4091e27dfb 00007f4091eb6559 00007f4091eb61e5 00007f4091eb5740 00007f4094ba8578 00007f4094ba8cd8 00007f4094b806e8 00007f4094b7baea 00007f4094b7af36
0809 07:33:10.305585 (+ 44713us) raft_consensus.cc:1671] finished
0809 07:33:10.305607 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:10.305807 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29510,"raft.queue_time_us":11970,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":2719,"spinlock_wait_cycles":17265280,"thread_start_us":128776,"threads_started":2,"wal-append.queue_time_us":311411}
W20260809 07:33:10.350438 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 195508us
I20260809 07:33:10.354837 32717 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
W20260809 07:33:10.346692 1262 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
W20260809 07:33:10.412801 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 59814.5us
W20260809 07:33:10.419976 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 159257us
I20260809 07:33:10.334529 1197 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.355239 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:10.257174 1224 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.346628 1211 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:10.454099 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:10.436975 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 81831.6us
I20260809 07:33:10.421660 698 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.457403 31005 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:10.331092 854 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.464694 32717 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:10.465617 32717 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.336673 1183 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.451993 936 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:10.462044 1193 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:10.452458 951 raft_consensus.cc:686] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.489552 32717 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:10.465000 603 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.495136 32717 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:10.474165 1199 raft_consensus.cc:686] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.530778 951 consensus_queue.cc:237] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.479718 1283 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.423100 1034 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.346892 1246 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.465850 698 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.489799 1049 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:10.540189 31175 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:10.546347 32717 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:10.556087 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 64837.6us
I20260809 07:33:10.557085 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:10.557909 31175 raft_consensus.cc:2396] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:10.558950 31175 raft_consensus.cc:3032] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:10.559242 31175 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.494407 1274 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.559890 1274 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.560485 1049 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:10.559281 31172 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [2.3-4.4]
I20260809 07:33:10.560107 31175 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.453554 1197 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.562536 31175 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.561630 1049 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:10.563761 1197 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:10.564453 1197 consensus_queue.cc:1046] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:10.535542 31170 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:10.566490 31174 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:10.572365 31170 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:10.572705 1014 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.573144 1014 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.562062 1274 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:10.580915 1083 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:10.599351 1014 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:10.613186 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 121305us
I20260809 07:33:10.461104 1211 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.464957 597 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.494148 936 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.646984 602 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:10.552145 1287 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:10.613692 1137 log.cc:924] Time spent T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Append to log took a long time: real 0.267s user 0.001s sys 0.000s
I20260809 07:33:10.519057 603 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.473807 664 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.442171 1222 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:10.713915 31175 raft_consensus.cc:1261] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:10.721478 1307 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:10.685719 1176 log.cc:924] Time spent T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.127s user 0.000s sys 0.013s
W20260809 07:33:10.723268 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 166339us
I20260809 07:33:10.673301 699 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.602201 1014 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.687130 1209 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:10.746084 31175 pending_rounds.cc:76] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 2
I20260809 07:33:10.752110 31045 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:10.759371 31175 pending_rounds.cc:97] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 07:33:10.759650 31175 raft_consensus.cc:2866] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:10.721959 1243 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.195s
W20260809 07:33:10.750372 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 175091us
I20260809 07:33:10.744107 1193 raft_consensus.cc:686] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.765923 31174 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:10.552848 1283 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.777801 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.782833 31174 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.727330 602 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.789543 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:10.791121 698 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:10.830834 1268 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.335s user 0.000s sys 0.001s
I20260809 07:33:10.831410 1277 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:10.727209 1199 consensus_queue.cc:237] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.831854 31049 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.766974 854 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.794661 31006 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:10.780774 1183 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.741838 1287 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:10.798441 1292 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
I20260809 07:33:10.797703 1193 consensus_queue.cc:237] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.799839 1283 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:10.761334 1014 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:10.796012 1231 consensus_meta.cc:287] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.718s user 0.011s sys 0.000s
I20260809 07:33:10.833441 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:10.879216 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 45875.6us
W20260809 07:33:10.804167 1258 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.374s user 0.001s sys 0.000s
I20260809 07:33:10.755160 1307 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.871528 31049 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:10.881305 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:10.804827 1054 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.885174 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:10.866856 1290 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:10.867774 1137 log.cc:924] Time spent T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260809 07:33:10.886003 32717 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:10.894460 32717 consensus_queue.cc:1225] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:10.894933 32717 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:10.895270 32717 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:10.891723 1310 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:10.884891 1231 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:10.892341 1300 log.cc:924] Time spent T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260809 07:33:10.885197 1229 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.893491 1214 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.891801 1303 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:10.784013 1209 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.898367 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:10.896009 32717 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.957726 32717 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:10.904865 31174 raft_consensus.cc:2396] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:10.880015 823 raft_consensus.cc:3032] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:10.879472 1283 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.885735 699 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
W20260809 07:33:10.890513 1213 consensus_meta.cc:287] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.619s user 0.010s sys 0.000s
I20260809 07:33:10.888578 1297 raft_consensus.cc:3032] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:10.992457 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:10.975940 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 518595us
I20260809 07:33:10.832381 655 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.016112 1295 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.019714 1243 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:10.927915 1310 raft_consensus.cc:686] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:11.024003 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:11.055106 31174 raft_consensus.cc:3032] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:11.065851 31174 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.962128 602 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:10.899220 1054 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.963320 823 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:11.066725 31174 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.081290 31174 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:10.832003 1277 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:10.961014 936 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.082415 936 consensus_queue.cc:1046] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:11.083438 936 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 07:33:11.110052 1301 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.239s user 0.001s sys 0.002s
I20260809 07:33:11.054207 597 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.079314 602 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.110904 597 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.111224 602 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.113020 597 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.027451 1213 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 07:33:11.113322 1273 log.cc:924] Time spent T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.101s user 0.000s sys 0.000s
I20260809 07:33:11.113528 597 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.114156 1287 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.069288 1034 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.068293 1295 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.114925 1034 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.116403 1295 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.116917 1034 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.073292 1243 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.022559 1297 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:10.894541 1290 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:11.123950 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 100038us
I20260809 07:33:11.027683 603 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.125216 31005 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:11.131404 31005 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:11.127002 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 474431us
I20260809 07:33:11.068202 699 raft_consensus.cc:686] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:11.136914 1269 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260809 07:33:11.137436 699 consensus_queue.cc:237] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:11.123217 1225 consensus_meta.cc:287] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 1.025s user 0.004s sys 0.008s
I20260809 07:33:11.079116 1050 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.022704 1283 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.113174 602 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.135870 1310 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
W20260809 07:33:11.140102 807 consensus_meta.cc:287] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.928s user 0.007s sys 0.003s
I20260809 07:33:10.963878 1214 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.173085 1214 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.113993 597 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.151880 32717 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:11.113719 1285 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:11.157375 31175 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:11.183559 1054 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.168459 807 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:11.170493 1231 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:11.184466 31118 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 58200.5us
I20260809 07:33:11.188169 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.143652 31005 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:11.168512 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.168916 1209 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.170203 655 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.159422 1226 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.171023 1367 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.189795 1297 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.190523 1367 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.191838 1243 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.139467 1213 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:11.117328 823 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:11.175979 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 866) took 1205 ms. Trace:
I20260809 07:33:10.991978 1303 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:11.177023 1306 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.150s user 0.000s sys 0.000s
I20260809 07:33:11.178025 1307 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.179342 32717 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:11.201380 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.201894 32717 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:11.201421 1014 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.203832 1014 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:10.956597 1229 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.190531 1226 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.195282 31177 rpcz_store.cc:276] 0809 07:33:09.968903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:09.969029 (+ 126us) service_pool.cc:227] Handling call
0809 07:33:09.969387 (+ 358us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:09.969788 (+ 401us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:10.053179 (+ 83391us) log.cc:842] Serialized 14 byte log entry
0809 07:33:10.571717 (+518538us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b1800379de0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7
0809 07:33:10.992362 (+420645us) log.cc:842] Serialized 14 byte log entry
0809 07:33:11.017124 (+ 24762us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:11.018025 (+ 901us) log.cc:842] Serialized 52 byte log entry
0809 07:33:11.025734 (+ 7709us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:11.030052 (+ 4318us) log.cc:842] Serialized 14 byte log entry
0809 07:33:11.073907 (+ 43855us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:11.117907 (+ 44000us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:11.174390 (+ 56483us) raft_consensus.cc:1671] finished
0809 07:33:11.174411 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:11.174546 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":181689,"prepare.queue_time_us":455082,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":1465,"raft.queue_time_us":266075,"raft.run_cpu_time_us":443,"raft.run_wall_time_us":73049,"spinlock_wait_cycles":13139328,"thread_start_us":726504,"threads_started":4,"wal-append.queue_time_us":149162}
I20260809 07:33:11.146468 1034 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.207370 1034 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.207963 1303 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.208267 1226 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.153769 1050 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.214143 1034 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.217046 1211 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.196352 1367 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:11.217490 1049 consensus_meta.cc:287] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.654s user 0.007s sys 0.001s
I20260809 07:33:11.201526 1214 consensus_queue.cc:1046] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260809 07:33:11.219872 1229 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.188068 1285 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.185921 1283 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.204252 1014 raft_consensus.cc:686] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:11.221603 1050 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.184360 1331 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:11.224655 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 907) took 1073 ms. Trace:
W20260809 07:33:11.226147 1201 consensus_meta.cc:287] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 1.128s user 0.010s sys 0.001s
I20260809 07:33:11.204286 597 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.217906 1049 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:11.232609 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 98012.7us
I20260809 07:33:11.235139 31169 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:11.206282 1307 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.244515 1307 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.237200 1331 raft_consensus.cc:686] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:11.246735 1307 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:11.247740 827 consensus_meta.cc:287] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.960s user 0.011s sys 0.000s
I20260809 07:33:11.226168 1137 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:33:11.230582 31165 rpcz_store.cc:276] 0809 07:33:10.150861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:10.151013 (+ 152us) service_pool.cc:227] Handling call
0809 07:33:10.154592 (+ 3579us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:10.422803 (+268211us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:10.422839 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:10.428886 (+ 6047us) log.cc:842] Serialized 26 byte log entry
0809 07:33:10.604799 (+175913us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:10.779667 (+174868us) log.cc:842] Serialized 14 byte log entry
0809 07:33:10.779852 (+ 185us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:10.779918 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:11.224431 (+444513us) raft_consensus.cc:1671] finished
0809 07:33:11.224456 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:11.224597 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":117918,"prepare.queue_time_us":290684,"prepare.run_wall_time_us":9551,"raft.queue_time_us":243130,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":56575,"spinlock_wait_cycles":13404416,"thread_start_us":414713,"threads_started":3,"wal-append.queue_time_us":418475}
I20260809 07:33:11.242460 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 915) took 1064 ms. Trace:
I20260809 07:33:11.243996 1049 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.249302 31170 rpcz_store.cc:276] 0809 07:33:10.177823 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:10.577322 (+399499us) service_pool.cc:227] Handling call
0809 07:33:10.577610 (+ 288us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:10.588187 (+ 10577us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:10.588217 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:10.856046 (+267829us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b2c000425a8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:33:10.868281 (+ 12235us) log.cc:842] Serialized 52 byte log entry
0809 07:33:11.069641 (+201360us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:11.116935 (+ 47294us) log.cc:842] Serialized 14 byte log entry
0809 07:33:11.117245 (+ 310us) log.cc:842] Serialized 14 byte log entry
0809 07:33:11.117391 (+ 146us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:11.117454 (+ 63us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:11.240148 (+122694us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b48000d0758. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:11.242208 (+ 2060us) raft_consensus.cc:1671] finished
0809 07:33:11.242229 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:11.242392 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":151146,"prepare.queue_time_us":47884,"prepare.run_wall_time_us":37,"raft.queue_time_us":154725,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":18860,"spinlock_wait_cycles":95093376,"thread_start_us":157279,"threads_started":1,"wal-append.queue_time_us":358630}
I20260809 07:33:11.253408 31181 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:11.255241 1137 tablet_bootstrap.cc:492] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Bootstrap complete.
I20260809 07:33:11.228915 1290 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.231947 32717 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.277446 1201 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
W20260809 07:33:11.293797 1201 raft_consensus.cc:1054] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Unable to promote non-voter 28b9965e21374d26b0ac2014c9bafdff: Illegal state: Leader has not yet committed an operation in its own term
I20260809 07:33:11.295085 1285 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.278594 1014 consensus_queue.cc:237] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.246022 1331 consensus_queue.cc:237] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.299839 1213 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:11.334626 30898 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:11.335114 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 22) took 3260 ms. Trace:
I20260809 07:33:11.335551 30744 rpcz_store.cc:276] 0809 07:33:08.075294 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:08.075701 (+ 407us) service_pool.cc:227] Handling call
0809 07:33:11.335042 (+3259341us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:10.111222 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:10.137568 (+ 26346us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:10.140153 (+ 2585us) write_op.cc:435] Acquired schema lock
0809 07:33:10.140174 (+ 21us) tablet.cc:660] Decoding operations
0809 07:33:10.140537 (+ 363us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:10.140591 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:10.140619 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:33:10.140820 (+ 201us) tablet.cc:699] Row locks acquired
0809 07:33:10.140834 (+ 14us) write_op.cc:260] PREPARE: finished
0809 07:33:10.140974 (+ 140us) write_op.cc:270] Start()
0809 07:33:10.141081 (+ 107us) write_op.cc:276] Timestamp: P: 1786260790140930 usec, L: 0
0809 07:33:10.141096 (+ 15us) op_driver.cc:331] REPLICATION: starting
0809 07:33:10.141727 (+ 631us) log.cc:842] Serialized 594 byte log entry
0809 07:33:10.423317 (+281590us) op_driver.cc:444] REPLICATION: finished
0809 07:33:10.726257 (+302940us) write_op.cc:301] APPLY: starting
0809 07:33:10.742708 (+ 16451us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:10.756879 (+ 14171us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:10.775563 (+ 18684us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:10.794386 (+ 18823us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:10.853005 (+ 58619us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:33:10.858914 (+ 5909us) write_op.cc:312] APPLY: finished
0809 07:33:10.885459 (+ 26545us) log.cc:842] Serialized 22 byte log entry
0809 07:33:10.903404 (+ 17945us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:10.917501 (+ 14097us) write_op.cc:454] Released schema lock
0809 07:33:10.927397 (+ 9896us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:11.003745 (+ 76348us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:11.023436 (+ 19691us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8479,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":235566,"apply.run_cpu_time_us":2812,"apply.run_wall_time_us":366564,"num_ops":1,"prepare.queue_time_us":116190,"prepare.run_cpu_time_us":2567,"prepare.run_wall_time_us":48934,"replication_time_us":215120,"thread_start_us":123448,"threads_started":2}]]}
I20260809 07:33:11.342525 1137 ts_tablet_manager.cc:1397] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Time spent bootstrapping tablet: real 1.907s user 0.022s sys 0.032s
I20260809 07:33:11.347644 1277 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.364104 30891 raft_consensus.cc:1261] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:11.366076 32717 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.379562 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.379886 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.380491 30891 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:11.390709 30896 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:11.391422 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:11.392083 30899 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:11.392206 30906 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:11.401813 30899 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:11.406522 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:11.407853 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:11.407995 1274 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.408556 31057 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:11.417416 30890 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:11.430572 31174 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:11.434422 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:11.434572 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.521s user 0.010s sys 0.037s
I20260809 07:33:11.434551 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:11.435065 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.522s user 0.010s sys 0.037s
W20260809 07:33:11.435385 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 522324us
I20260809 07:33:11.433130 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.437170 30906 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:11.410004 1007 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.443485 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.444334 30898 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:11.443233 951 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.446419 951 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.448313 951 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.449571 1297 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:11.441322 1246 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:11.410938 1274 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.458525 30898 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:11.469187 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.469924 30906 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:11.473445 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.474052 30906 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:33:11.474462 1274 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.482559 30890 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 07:33:11.443994 1007 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.485073 1007 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.487751 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.501754 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.497835 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.503046 30890 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:11.504305 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.505008 31174 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:11.504941 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.510175 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.512620 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.513332 30890 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:11.514782 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.525758 31175 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 2.
I20260809 07:33:11.562675 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.564515 31175 raft_consensus.cc:2396] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260809 07:33:11.568174 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:11.569222 31175 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:11.570004 30891 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:11.572248 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.573072 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.573800 30890 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:11.575166 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:11.575281 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:11.575843 31170 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:11.575982 30890 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:11.573129 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.587630 31169 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7.
W20260809 07:33:11.565883 1267 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.871s user 0.015s sys 0.018s
I20260809 07:33:11.516004 936 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.589926 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.567538 1274 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:11.593102 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.593257 31170 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:11.593600 30906 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260809 07:33:11.594642 1267 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s user 0.015s sys 0.018s
W20260809 07:33:11.593606 1171 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.061s user 0.005s sys 0.010s
I20260809 07:33:11.516008 1083 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260809 07:33:11.594676 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.492969 1222 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260809 07:33:11.644558 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.642660 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.645874 31170 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:11.634109 1201 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:11.635831 802 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.649814 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.636355 597 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.654781 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:11.654917 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 60297.6us
I20260809 07:33:11.563282 1137 raft_consensus.cc:348] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:11.658067 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:11.641243 1171 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.109s user 0.005s sys 0.010s
W20260809 07:33:11.643055 875 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s user 0.005s sys 0.035s
I20260809 07:33:11.588212 1193 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260809 07:33:11.671576 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.671098 802 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.708881 802 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.709077 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.710520 31182 raft_consensus.cc:3032] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:11.710852 31182 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:11.710893 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.711687 31182 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:11.712921 31182 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:11.657534 30906 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:11.684762 1225 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:11.714712 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.685520 597 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.714700 1225 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:11.725162 1225 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:11.634729 807 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260809 07:33:11.648092 32198 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 5 LEARNER]: Advancing to term 6
W20260809 07:33:11.729957 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 72003.7us
I20260809 07:33:11.595197 936 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.637753 1274 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.733582 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.733880 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:11.727730 875 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s user 0.006s sys 0.035s
I20260809 07:33:11.739907 31182 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260809 07:33:11.728050 1281 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.762699 31182 pending_rounds.cc:76] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:11.765623 31182 pending_rounds.cc:97] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260809 07:33:11.765910 31182 raft_consensus.cc:2866] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:11.739852 31175 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:11.743840 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.728222 1201 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:11.731827 1183 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.774295 31169 raft_consensus.cc:2396] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260809 07:33:11.771121 30745 catalog_manager.cc:5495] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:11.732030 1137 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 28b9965e21374d26b0ac2014c9bafdff, State: Initialized, Role: LEARNER
I20260809 07:33:11.779781 30745 catalog_manager.cc:5495] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:33:11.779690 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46193us
I20260809 07:33:11.764503 807 consensus_queue.cc:1225] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer 2c9de56c3fa3487c9dac990675c34be5 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260809 07:33:11.788805 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.785799 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:11.771103 1208 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.673s user 0.006s sys 0.014s
I20260809 07:33:11.727373 1267 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
W20260809 07:33:11.778928 699 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.009s sys 0.023s
I20260809 07:33:11.783344 30745 catalog_manager.cc:5495] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:11.796470 31120 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.824019 30853 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.776314 1230 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.825450 30745 catalog_manager.cc:5495] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 2 to 5, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:33:11.787496 1327 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.847s user 0.000s sys 0.038s
I20260809 07:33:11.832598 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.833096 30745 catalog_manager.cc:5495] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:11.784467 1295 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:33:11.795761 1208 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.986s user 0.008s sys 0.014s
W20260809 07:33:11.826438 699 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s user 0.013s sys 0.023s
I20260809 07:33:11.831785 1222 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:11.804581 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.741s user 0.016s sys 0.020s
I20260809 07:33:11.774497 597 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.852249 664 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:11.773670 1281 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.858518 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.859285 30853 leader_election.cc:398] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7.
I20260809 07:33:11.914115 30853 leader_election.cc:401] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:11.822623 1373 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:11.879074 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46563us
I20260809 07:33:11.875677 30745 catalog_manager.cc:5495] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:11.830601 1193 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:11.787706 1195 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:11.835094 1230 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.888270 1337 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:11.862289 1295 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:11.889055 1278 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.240s user 0.010s sys 0.019s
I20260809 07:33:11.895818 1320 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:11.792732 1277 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.084s
I20260809 07:33:11.861486 802 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:11.852541 1291 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260809 07:33:11.906484 1274 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.852414 1137 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:11.906899 1171 raft_consensus.cc:3032] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:11.961362 1171 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:11.907539 936 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.861593 1197 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:11.853142 1365 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:11.790894 1183 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.962386 1197 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.964023 1183 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.910982 1222 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:11.964514 1197 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:11.964648 1222 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:11.911345 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 922) took 1139 ms. Trace:
I20260809 07:33:11.967582 1230 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.861317 1201 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.962466 1365 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:11.971517 1372 log.cc:924] Time spent T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260809 07:33:11.970274 1365 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:11.915653 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:11.916316 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 60748.4us
I20260809 07:33:11.918850 30745 catalog_manager.cc:5495] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:33:11.920769 31006 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.716s user 0.002s sys 0.029s
W20260809 07:33:11.977468 31006 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.775s user 0.002s sys 0.030s
I20260809 07:33:11.981645 1331 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:11.982263 1331 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.921258 597 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 7
I20260809 07:33:11.914964 664 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:11.920502 1193 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:11.951741 1278 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.456s user 0.011s sys 0.019s
I20260809 07:33:11.952751 802 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:11.912168 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.848s user 0.016s sys 0.023s
W20260809 07:33:11.983026 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 67451.7us
I20260809 07:33:11.985054 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:11.985715 30896 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s user 0.022s sys 0.071s
I20260809 07:33:11.985137 30745 catalog_manager.cc:5495] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:11.982019 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:11.858067 1327 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.968s user 0.006s sys 0.038s
I20260809 07:33:11.988427 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:11.951474 1295 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:11.952647 1320 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:11.944597 1337 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:11.966480 31164 rpcz_store.cc:276] 0809 07:33:10.772288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.058365 (+286077us) service_pool.cc:227] Handling call
0809 07:33:11.058756 (+ 391us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:11.170824 (+112068us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:11.185696 (+ 14872us) log.cc:842] Serialized 14 byte log entry
0809 07:33:11.655433 (+469737us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:11.665678 (+ 10245us) log.cc:842] Serialized 26 byte log entry
0809 07:33:11.665820 (+ 142us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:11.666879 (+ 1059us) log.cc:842] Serialized 14 byte log entry
0809 07:33:11.684984 (+ 18105us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:11.732585 (+ 47601us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b5400186354. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:11.732635 (+ 50us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:11.904241 (+171606us) spinlock_profiling.cc:243] Waited 91.5 ms on lock 0x7b48000d0758. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:11.904984 (+ 743us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b180037cf60. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f
0809 07:33:11.905448 (+ 464us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b48000d0758. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:11.911115 (+ 5667us) raft_consensus.cc:1671] finished
0809 07:33:11.911135 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:11.911271 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":278746,"prepare.queue_time_us":20769,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":138215,"raft.run_cpu_time_us":5474,"raft.run_wall_time_us":536429,"spinlock_wait_cycles":437812864,"thread_start_us":464408,"threads_started":1,"wal-append.queue_time_us":723024}
I20260809 07:33:12.032908 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:11.998577 31057 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260809 07:33:12.087541 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.033329 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:12.088668 31054 raft_consensus.cc:2396] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260809 07:33:12.034559 30896 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.566s user 0.022s sys 0.074s
I20260809 07:33:12.026062 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.024435 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.088779 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.089942 31058 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:12.025444 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:12.035802 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 53857.8us
I20260809 07:33:12.090988 30853 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:11.924670 1195 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.002810 1183 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.070955 31048 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:12.086082 1320 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:12.097539 1392 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:12.067255 936 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:12.061487 1383 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:12.061527 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.065650 823 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:11.962620 1332 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.041745 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:12.002677 31045 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260809 07:33:12.034907 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 971064us
I20260809 07:33:12.180945 1332 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:11.994586 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.999073 31049 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:12.084534 30745 catalog_manager.cc:5495] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 2, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 2 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:33:11.994144 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.761s user 0.012s sys 0.031s
I20260809 07:33:12.021103 664 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:12.095081 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:12.032480 31059 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 07:33:12.095404 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:11.920359 1373 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.097365 31175 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:12.069300 1197 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:12.037083 1193 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:12.036289 802 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.070366 31051 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:11.962280 1171 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.103075 31054 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:12.086171 1337 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:12.091212 1014 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s user 0.033s sys 0.098s
W20260809 07:33:12.230751 1014 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s user 0.033s sys 0.099s
I20260809 07:33:12.109549 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.124847 31169 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260809 07:33:12.234952 1379 log.cc:924] Time spent T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260809 07:33:12.235569 1171 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.125777 936 raft_consensus.cc:686] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:12.100713 854 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260809 07:33:12.122829 1383 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:12.102684 1392 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:12.102828 1277 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.180351 951 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.106812 1291 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:12.179874 31046 raft_consensus.cc:1226] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:12.185436 31057 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:11.998785 31055 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
W20260809 07:33:12.186506 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 152232us
W20260809 07:33:12.190416 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.990s user 0.013s sys 0.033s
I20260809 07:33:12.101950 1183 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.253170 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.253854 31059 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:12.254416 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:12.255053 30904 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:12.197588 30745 catalog_manager.cc:5495] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.188714 30852 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:12.203651 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 108658us
W20260809 07:33:12.192355 1246 consensus_meta.cc:287] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.734s user 0.011s sys 0.000s
I20260809 07:33:12.209451 31172 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:12.210927 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.191730 1274 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:12.212903 32198 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 LEARNER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260809 07:33:12.206347 1361 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:12.287846 31051 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260809 07:33:12.222867 1374 log.cc:924] Time spent T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.124s user 0.000s sys 0.000s
I20260809 07:33:12.291107 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.185446 1281 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.256943 30745 catalog_manager.cc:5495] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.261617 31054 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:12.262187 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 1.11853e+06us
I20260809 07:33:12.262893 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.294994 31055 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 2.
I20260809 07:33:12.265560 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.268985 30906 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:12.270448 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:12.270653 31057 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260809 07:33:12.271754 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 83116.4us
I20260809 07:33:12.267908 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:12.266441 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.249316 1392 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.258833 951 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.276515 936 consensus_queue.cc:237] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.279346 1373 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.249647 1277 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.318162 1277 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.319952 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.296018 31048 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:33:12.282832 1377 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.092s user 0.000s sys 0.000s
I20260809 07:33:12.283437 1195 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.285146 1295 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.244789 1383 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:12.343186 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.344861 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.286715 1230 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
W20260809 07:33:12.289402 30891 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s user 0.000s sys 0.043s
W20260809 07:33:12.348058 30891 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.704s user 0.001s sys 0.043s
I20260809 07:33:12.271145 32198 pending_rounds.cc:76] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Aborting all ops after (but not including) 3
I20260809 07:33:12.299853 30745 catalog_manager.cc:5495] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.302477 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.289435 699 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.304371 1222 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.349746 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:12.350286 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:12.305718 1225 consensus_meta.cc:287] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.580s user 0.013s sys 0.000s
I20260809 07:33:12.306288 1331 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.260722 1291 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.308471 951 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.290616 1014 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.269404 1274 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:12.310616 1183 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.322132 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:12.357772 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.325989 30852 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:12.286167 664 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.360004 31169 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:12.377594 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:12.346992 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.349728 1332 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.298100 1361 raft_consensus.cc:686] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:12.355115 30745 catalog_manager.cc:5495] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.384085 30745 catalog_manager.cc:5495] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.356096 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.328183 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.355312 31051 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:12.363942 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.386569 31059 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:12.355911 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.357880 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.354275 32198 pending_rounds.cc:97] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260809 07:33:12.356375 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.379360 31048 raft_consensus.cc:2396] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260809 07:33:12.387912 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 61992.2us
I20260809 07:33:12.369146 1274 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.396628 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.396430 31046 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:12.353657 1387 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.353865 1246 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.382405 1230 raft_consensus.cc:686] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:12.348382 1320 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.366835 1225 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.368968 1267 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:12.375936 951 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260809 07:33:12.356962 1384 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.350890 1281 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:12.405365 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 83319.4us
I20260809 07:33:12.366636 1310 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260809 07:33:12.395346 32198 raft_consensus.cc:2866] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:12.412621 1230 consensus_queue.cc:237] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.410032 1007 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.437668 31005 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:12.414878 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.395525 568 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.346176 1337 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.438889 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.385444 31057 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:12.393024 30745 catalog_manager.cc:5495] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.440608 30745 catalog_manager.cc:5495] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 1 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.416510 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.420583 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:12.443192 31055 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.421175 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.397006 1331 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260809 07:33:12.424696 1327 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.470847 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.471850 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.356614 699 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.429347 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:12.369076 1014 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:12.435012 30898 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.011s sys 0.041s
I20260809 07:33:12.398952 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:12.471935 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.417704 1267 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:12.453985 30745 catalog_manager.cc:5495] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:12.418362 802 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.408516 1387 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.433835 1361 consensus_queue.cc:237] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.457911 1222 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.431185 597 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7.
I20260809 07:33:12.461037 1337 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.468250 568 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.469381 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.470161 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:12.477883 30898 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s user 0.012s sys 0.041s
I20260809 07:33:12.478045 31051 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:12.411020 1246 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:12.455951 31055 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:12.461946 1383 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.432030 1281 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.473963 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.428618 1384 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.503407 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:12.662168 1337 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.662477 1297 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:12.663079 1297 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.662724 1337 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.665019 1297 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.666249 1337 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.667856 1387 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.475097 854 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.669216 854 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.671667 568 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.672605 31169 raft_consensus.cc:2396] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:12.673415 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.674696 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.675797 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.677153 31051 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:12.642683 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.497545 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.473006 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.683073 31049 raft_consensus.cc:2379] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:12.639055 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:12.640027 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 202425us
I20260809 07:33:12.683662 31045 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:12.685082 31046 raft_consensus.cc:2379] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:12.682386 31054 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:12.684466 31055 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:12.659843 31048 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
W20260809 07:33:12.551575 31174 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s user 0.036s sys 0.039s
W20260809 07:33:12.687805 31174 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s user 0.037s sys 0.044s
I20260809 07:33:12.690742 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:12.691409 31057 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:12.692600 1387 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:12.694036 1387 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:12.694298 1014 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.694923 1014 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:12.695475 1320 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.692924 1007 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:12.696768 31059 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:12.697467 1384 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.699232 854 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.699744 854 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.700158 854 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.688571 1347 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.701906 854 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.686759 568 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:12.702232 1347 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.702741 568 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.702368 1193 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.704670 568 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.704780 699 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.704929 31175 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:12.620464 32199 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 LEARNER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:12.707206 31059 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:12.660399 1267 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:12.449179 1365 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.611163 32198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 5) took 1053 ms. Trace:
I20260809 07:33:12.659303 1310 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:12.679718 1295 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.698143 1246 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:12.708180 1201 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s user 0.023s sys 0.022s
I20260809 07:33:12.698155 1291 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.525269 1274 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:12.747505 31165 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s user 0.012s sys 0.014s
W20260809 07:33:12.749707 31165 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s user 0.012s sys 0.015s
W20260809 07:33:12.744433 1201 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s user 0.023s sys 0.022s
W20260809 07:33:12.744421 1137 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.722s user 0.005s sys 0.022s
I20260809 07:33:12.745186 664 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.742690 1310 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.494959 1327 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:12.706929 32141 thread.cc:636] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.745s user 0.012s sys 0.024s
W20260809 07:33:12.780440 32141 thread.cc:602] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.817s user 0.012s sys 0.024s
I20260809 07:33:12.714231 32198 rpcz_store.cc:276] 0809 07:33:11.510913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.561504 (+ 50591us) service_pool.cc:227] Handling call
0809 07:33:11.592909 (+ 31405us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:12.496432 (+903523us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.527709 (+ 31277us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:12.715826 1365 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.782720 1365 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:12.773119 1137 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.751s user 0.005s sys 0.022s
I20260809 07:33:12.784606 1137 ts_tablet_manager.cc:1428] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Time spent starting tablet: real 1.439s user 0.009s sys 0.035s
W20260809 07:33:12.784916 1137 ts_tablet_manager.cc:1460] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Tablet startup took 1439ms
I20260809 07:33:12.787940 1383 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.795423 1274 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:12.798061 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 945) took 1088 ms. Trace:
I20260809 07:33:12.750836 1225 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:12.817147 30896 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s user 0.003s sys 0.024s
W20260809 07:33:12.817659 30896 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s user 0.004s sys 0.024s
I20260809 07:33:12.798516 31170 rpcz_store.cc:276] 0809 07:33:11.709662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.764155 (+ 54493us) service_pool.cc:227] Handling call
0809 07:33:11.764559 (+ 404us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:11.764920 (+ 361us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:33:11.775666 (+ 10746us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.247678 (+472012us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:12.256284 (+ 8606us) log.cc:842] Serialized 26 byte log entry
0809 07:33:12.256408 (+ 124us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:12.256653 (+ 245us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.299937 (+ 43284us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b18001629a0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c990 00007f4091fce864 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7 00007f4094f14058
0809 07:33:12.300144 (+ 207us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.300211 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:12.797807 (+497596us) raft_consensus.cc:1671] finished
0809 07:33:12.797830 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.797999 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":75514,"prepare.queue_time_us":19457,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":34892,"raft.queue_time_us":59712,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":18374,"spinlock_wait_cycles":4825344,"thread_start_us":460126,"threads_started":1,"wal-append.queue_time_us":938313}
I20260809 07:33:12.820425 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 941) took 1112 ms. Trace:
I20260809 07:33:12.829444 802 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.830459 1281 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:12.835618 1392 consensus_meta.cc:287] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.532s user 0.009s sys 0.000s
I20260809 07:33:12.820043 1347 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.825111 31177 rpcz_store.cc:276] 0809 07:33:11.708437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.708575 (+ 138us) service_pool.cc:227] Handling call
0809 07:33:11.708969 (+ 394us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:11.731173 (+ 22204us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:11.731203 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:11.731607 (+ 404us) log.cc:842] Serialized 26 byte log entry
0809 07:33:11.907602 (+175995us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b48000d0758. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:12.186000 (+278398us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:12.214236 (+ 28236us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.247810 (+ 33574us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.247886 (+ 76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:12.798336 (+550450us) raft_consensus.cc:1671] finished
0809 07:33:12.820199 (+ 21863us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.820358 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":91847,"prepare.queue_time_us":27976,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":69500,"raft.run_cpu_time_us":1436,"raft.run_wall_time_us":196782,"spinlock_wait_cycles":67385728,"thread_start_us":409071,"threads_started":1,"wal-append.queue_time_us":921549}
W20260809 07:33:12.858716 30899 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.912s user 0.018s sys 0.019s
W20260809 07:33:12.859274 30899 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.913s user 0.019s sys 0.019s
I20260809 07:33:12.862690 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 959) took 1403 ms. Trace:
I20260809 07:33:12.816893 1193 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:12.864008 31058 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.737s user 0.004s sys 0.026s
W20260809 07:33:12.864343 31058 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.738s user 0.004s sys 0.026s
W20260809 07:33:12.841703 1278 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s user 0.007s sys 0.020s
I20260809 07:33:12.863261 30896 rpcz_store.cc:276] 0809 07:33:11.458915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.459040 (+ 125us) service_pool.cc:227] Handling call
0809 07:33:11.468277 (+ 9237us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:11.468534 (+ 257us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:12.089474 (+620940us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.862101 (+772627us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:12.862173 (+ 72us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:12.862224 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.862353 (+ 129us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.862568 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":709011,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":44192,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":1348865,"threads_started":2}
W20260809 07:33:12.878574 31120 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.66:36075 timed out after 1.639s (SENT) [suppressed 14 similar messages]
W20260809 07:33:12.881181 1278 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s user 0.007s sys 0.020s
I20260809 07:33:12.836025 1392 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.864611 1391 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:12.899127 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 957) took 1441 ms. Trace:
I20260809 07:33:12.900430 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:12.903170 30896 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:12.902056 30899 rpcz_store.cc:276] 0809 07:33:11.458130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.459216 (+ 1086us) service_pool.cc:227] Handling call
0809 07:33:11.471526 (+ 12310us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:11.472412 (+ 886us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:11.472441 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:11.490260 (+ 17819us) log.cc:842] Serialized 52 byte log entry
0809 07:33:11.917018 (+426758us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:12.860043 (+943025us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.898760 (+ 38717us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.898856 (+ 96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:12.898898 (+ 42us) raft_consensus.cc:1671] finished
0809 07:33:12.898909 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.899062 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39441,"prepare.queue_time_us":331919,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":50292,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":1287537,"threads_started":2,"wal-append.queue_time_us":791893}
W20260809 07:33:12.900472 936 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s user 0.013s sys 0.010s
W20260809 07:33:12.911825 936 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s user 0.014s sys 0.010s
I20260809 07:33:12.913218 1365 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:12.913859 1365 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:12.915582 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 948) took 1075 ms. Trace:
I20260809 07:33:12.915876 1365 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:12.915961 31174 rpcz_store.cc:276] 0809 07:33:11.840449 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.862191 (+ 21742us) service_pool.cc:227] Handling call
0809 07:33:11.887812 (+ 25621us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:11.903206 (+ 15394us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:11.903242 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:11.903692 (+ 450us) log.cc:842] Serialized 26 byte log entry
0809 07:33:12.688228 (+784536us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:12.688270 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.688344 (+ 74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:12.915358 (+227014us) raft_consensus.cc:1671] finished
0809 07:33:12.915380 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.915523 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14973,"raft.queue_time_us":15563,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":784154,"threads_started":1,"wal-append.queue_time_us":985992}
I20260809 07:33:12.927990 1391 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:12.928473 1391 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:12.930707 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:12.931841 30899 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:12.940331 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 949) took 1086 ms. Trace:
I20260809 07:33:12.940723 31165 rpcz_store.cc:276] 0809 07:33:11.853998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.855221 (+ 1223us) service_pool.cc:227] Handling call
0809 07:33:11.861675 (+ 6454us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:11.924694 (+ 63019us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:11.924725 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:11.949878 (+ 25153us) log.cc:842] Serialized 26 byte log entry
0809 07:33:12.750124 (+800246us) raft_consensus.cc:1631] Marking committed up to 1
0809 07:33:12.753606 (+ 3482us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.866771 (+113165us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b54003a0cd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:12.866829 (+ 58us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:12.940026 (+ 73197us) raft_consensus.cc:1671] finished
0809 07:33:12.940048 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.940261 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":7512,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8041,"spinlock_wait_cycles":144091904,"thread_start_us":754758,"threads_started":1,"wal-append.queue_time_us":983358}
W20260809 07:33:12.946743 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.557s user 0.014s sys 0.011s
W20260809 07:33:12.947058 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.557s user 0.014s sys 0.011s
W20260809 07:33:12.947297 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 557610us
I20260809 07:33:12.948825 30853 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:12.950534 1195 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:12.950903 1195 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:12.951210 1195 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:12.951972 1398 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.310s user 0.001s sys 0.001s
I20260809 07:33:12.953732 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 924) took 1450 ms. Trace:
I20260809 07:33:12.954082 31181 rpcz_store.cc:276] 0809 07:33:11.503008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.504107 (+ 1099us) service_pool.cc:227] Handling call
0809 07:33:11.504413 (+ 306us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:11.562987 (+ 58574us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:11.563017 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:11.566911 (+ 3894us) log.cc:842] Serialized 26 byte log entry
0809 07:33:12.084039 (+517128us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:12.166817 (+ 82778us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.180012 (+ 13195us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.180094 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:12.951452 (+771358us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:12.953417 (+ 1965us) raft_consensus.cc:1671] finished
0809 07:33:12.953540 (+ 123us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.953671 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6575,"prepare.queue_time_us":92953,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":55549,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":5637,"spinlock_wait_cycles":76179456,"thread_start_us":446528,"threads_started":1,"wal-append.queue_time_us":860477}
I20260809 07:33:12.938130 30899 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:12.971624 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:12.972404 30899 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:12.976444 30896 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
W20260809 07:33:12.977700 30891 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s user 0.012s sys 0.016s
I20260809 07:33:12.977911 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:12.978842 30899 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:12.974455 30911 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s user 0.005s sys 0.023s
W20260809 07:33:12.976750 31172 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s user 0.002s sys 0.023s
W20260809 07:33:12.978535 30891 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s user 0.012s sys 0.016s
I20260809 07:33:12.980078 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:12.981002 31181 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:12.980437 31172 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s user 0.003s sys 0.023s
I20260809 07:33:12.983114 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.983379 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:12.983852 30896 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:12.984021 31182 raft_consensus.cc:2396] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:12.981514 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 974) took 1385 ms. Trace:
I20260809 07:33:12.985241 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:12.986776 30891 rpcz_store.cc:276] 0809 07:33:11.595935 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.636434 (+ 40499us) service_pool.cc:227] Handling call
0809 07:33:11.636815 (+ 381us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:11.637039 (+ 224us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:12.348423 (+711384us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.981207 (+632784us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:12.981258 (+ 51us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:12.981310 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.981366 (+ 56us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:12.981466 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":564415,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":259,"thread_start_us":1336301,"threads_started":2}
W20260809 07:33:12.983139 30911 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s user 0.006s sys 0.023s
W20260809 07:33:12.986388 30909 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.579s user 0.003s sys 0.033s
W20260809 07:33:12.977898 31006 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.587s user 0.006s sys 0.016s
W20260809 07:33:12.990691 31006 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.608s user 0.006s sys 0.017s
W20260809 07:33:12.992446 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 609604us
W20260809 07:33:12.992997 30909 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.582s user 0.003s sys 0.033s
W20260809 07:33:13.001142 30906 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s user 0.000s sys 0.038s
I20260809 07:33:13.003170 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.003804 31182 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 2 FOLLOWER]: Advancing to term 4
I20260809 07:33:13.004988 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:13.003578 30906 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s user 0.000s sys 0.038s
W20260809 07:33:13.030728 30849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.512s user 0.000s sys 0.022s
W20260809 07:33:13.012208 30900 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s user 0.007s sys 0.019s
W20260809 07:33:13.040208 30900 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s user 0.007s sys 0.022s
I20260809 07:33:13.040795 30890 raft_consensus.cc:595] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.041173 30890 raft_consensus.cc:603] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:13.006659 31177 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 07:33:13.034242 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.034168 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:13.037032 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 349659us
I20260809 07:33:13.043036 31177 pending_rounds.cc:76] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 1
I20260809 07:33:13.041519 31181 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:12.982125 1310 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.040704 30891 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:13.044878 31177 pending_rounds.cc:97] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 07:33:13.052350 31177 raft_consensus.cc:2866] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:13.052934 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.053699 30891 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.042765 30899 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:13.056504 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.059194 30899 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:13.032720 1327 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.067466 31005 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.079519 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 960) took 1619 ms. Trace:
I20260809 07:33:13.086611 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.082705 30898 rpcz_store.cc:276] 0809 07:33:11.460118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.460243 (+ 125us) service_pool.cc:227] Handling call
0809 07:33:11.460730 (+ 487us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:11.460921 (+ 191us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:11.917095 (+456174us) log.cc:842] Serialized 14 byte log entry
0809 07:33:12.678944 (+761849us) log.cc:842] Serialized 14 byte log entry
0809 07:33:13.065675 (+386731us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:13.065721 (+ 46us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:13.065778 (+ 57us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.079320 (+ 13542us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.079458 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31161,"prepare.queue_time_us":668236,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":645,"raft.queue_time_us":187685,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":23907,"thread_start_us":1566403,"threads_started":3,"wal-append.queue_time_us":917293}
I20260809 07:33:13.088369 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.083526 1387 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:13.050943 1365 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260809 07:33:13.084443 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.078047 1392 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.092451 30891 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
W20260809 07:33:12.785171 1137 ts_tablet_manager.cc:1462] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Trace:
0809 07:33:07.987539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:07.991203 (+ 3664us) service_pool.cc:227] Handling call
0809 07:33:08.102560 (+111357us) ts_tablet_manager.cc:928] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Initiating tablet copy from peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749)
0809 07:33:08.505417 (+402857us) tablet_copy_client.cc:361] Tablet copy session begun
0809 07:33:08.572916 (+ 67499us) tablet_metadata.cc:641] Metadata flushed
0809 07:33:08.573203 (+ 287us) tablet_copy_client.cc:490] Wrote new tablet metadata
0809 07:33:08.823491 (+250288us) tablet_copy_client.cc:411] Wrote new consensus metadata
0809 07:33:08.868643 (+ 45152us) tablet_replica.cc:205] Creating consensus instance
0809 07:33:08.869080 (+ 437us) ts_tablet_manager.cc:983] Replica registered.
0809 07:33:08.869138 (+ 58us) inbound_call.cc:173] Queueing success response
0809 07:33:09.243050 (+373912us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b18002d0100. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c628 00007f4091fd1002 00007f4099dee213 00007f4099deda4d 00007f4099e86a17 00007f4099eb7a8e 00007f4099e9d5ad
0809 07:33:09.430116 (+187066us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0809 07:33:11.345356 (+1915240us) ts_tablet_manager.cc:1429] Starting tablet replica
0809 07:33:11.348196 (+ 2840us) tablet_replica.cc:259] Starting instrumentation
0809 07:33:11.429944 (+ 81748us) tablet_replica.cc:278] Starting consensus
Metrics: {"dns_us":356,"mutex_wait_us":141306,"spinlock_wait_cycles":6531840,"tablet-copy.queue_time_us":102595,"tablet-download-pool.queue_time_us":33363,"tablet-download-pool.run_cpu_time_us":1328,"tablet-download-pool.run_wall_time_us":66044,"thread_start_us":870060,"threads_started":3}
I20260809 07:33:13.094635 30899 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:13.059294 1246 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260809 07:33:13.093679 30853 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:13.061985 1310 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260809 07:33:13.097513 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.096154 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.100379 1347 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:13.106360 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.111820 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.112006 31177 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:13.066380 699 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260809 07:33:13.111526 30891 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:13.106477 32199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59116 (request call id 10) took 1333 ms. Trace:
I20260809 07:33:13.107033 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:13.107748 1389 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.466s user 0.000s sys 0.001s
I20260809 07:33:13.108085 31165 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:13.110476 30899 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:13.116122 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.116082 31181 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:13.119108 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.107025 1392 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:13.119767 30744 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: config changed from index -1 to 3, term changed from 2 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to <none>, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New cstate: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:13.124956 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:13.125898 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46644.5us
I20260809 07:33:13.110657 30896 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:13.129351 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.130196 30896 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:13.131554 31182 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:13.111186 30898 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:13.137668 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.137665 31057 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 07:33:13.143873 31059 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:13.119177 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.149900 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.150507 30890 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:13.150916 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.150849 30852 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.152563 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.153744 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.114746 1373 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260809 07:33:13.097013 1267 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.157599 30852 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.127696 1392 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:13.163452 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:13.149122 1083 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.731s user 0.015s sys 0.023s
I20260809 07:33:13.166102 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.174854 31181 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:13.175055 30852 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.175112 31182 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:13.173864 30891 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:13.181188 30852 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.183138 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.184024 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.185497 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.155745 1230 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.177245 1347 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.187987 30852 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.189254 1332 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:13.212690 30890 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:13.163502 1365 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260809 07:33:13.170882 1083 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.761s user 0.015s sys 0.023s
I20260809 07:33:13.144739 1246 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260809 07:33:13.163831 854 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:13.194164 30891 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:13.115706 32199 rpcz_store.cc:276] 0809 07:33:11.775987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:11.784504 (+ 8517us) service_pool.cc:227] Handling call
0809 07:33:12.518357 (+733853us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:12.713058 (+194701us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.106215 (+393157us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54005109d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:13.106390 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1628309248}
I20260809 07:33:13.187186 31170 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:13.217569 1331 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:13.224007 1267 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:13.199903 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.225719 30899 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:13.226737 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.231442 1195 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.233371 1195 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.235416 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.236316 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 967) took 1132 ms. Trace:
I20260809 07:33:13.219340 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.236713 31058 rpcz_store.cc:276] 0809 07:33:12.083110 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:12.094995 (+ 11885us) service_pool.cc:227] Handling call
0809 07:33:12.103053 (+ 8058us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:12.103255 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:12.864697 (+761442us) log.cc:842] Serialized 14 byte log entry
0809 07:33:13.215444 (+350747us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:13.215500 (+ 56us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:13.215548 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.215628 (+ 80us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.215735 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1826,"prepare.queue_time_us":961540,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":22390,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2696,"thread_start_us":1066305,"threads_started":2}
I20260809 07:33:13.237973 30890 raft_consensus.cc:2362] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5. Re-sending same reply.
I20260809 07:33:13.238973 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.241173 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:13.179273 30904 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s user 0.000s sys 0.037s
I20260809 07:33:13.154953 1384 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.035s
I20260809 07:33:13.215029 30852 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.160307 597 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:13.192636 1327 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:13.218969 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.247736 30891 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.249693 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.250377 31182 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.212190 1439 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.245522 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.246852 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.252609 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.251286 30853 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:13.244455 30904 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s user 0.000s sys 0.037s
I20260809 07:33:13.258977 30891 raft_consensus.cc:2396] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:13.269467 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.269690 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.281584 30898 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:13.283054 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.284277 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.285382 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.286772 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.289047 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.290707 1384 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.291400 1384 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.292255 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.293455 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.288975 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.290227 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.296144 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.296325 1457 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:13.296911 1457 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:13.297284 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.298067 30900 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:13.294868 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.299510 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.299525 31005 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.300413 30911 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:13.302199 30911 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:13.302258 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.302826 30911 raft_consensus.cc:595] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.303817 31175 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260809 07:33:13.304457 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.304807 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:13.266538 1214 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s user 0.014s sys 0.017s
W20260809 07:33:13.269732 1370 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.667s user 0.030s sys 0.000s
I20260809 07:33:13.292281 30896 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:13.315953 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.247462 1327 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:13.287452 30906 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:13.254104 597 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.318286 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.319646 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.345592 31055 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:33:13.314210 30899 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:13.320739 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:13.321834 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 106851us
I20260809 07:33:13.315310 1384 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:13.339000 31054 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260809 07:33:13.339612 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.339802 1230 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:13.340284 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.365185 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.365811 31059 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.341504 31046 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:13.358909 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.342085 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.342250 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.342718 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.343366 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:13.343686 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 92491.5us
I20260809 07:33:13.269392 1183 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:13.348915 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.369112 31055 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3.
I20260809 07:33:13.306671 31182 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:13.349419 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 981) took 1089 ms. Trace:
I20260809 07:33:13.297821 1457 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.357831 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.361920 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.361022 31054 pending_rounds.cc:76] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:13.369506 30853 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:13.372511 30890 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:13.379660 30899 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:13.374230 31174 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:13.357924 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:13.332952 1214 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s user 0.014s sys 0.018s
W20260809 07:33:13.333324 1370 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.788s user 0.030s sys 0.000s
I20260809 07:33:13.374822 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:13.386372 30898 raft_consensus.cc:1226] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:13.380739 30896 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:13.381157 31054 pending_rounds.cc:97] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 07:33:13.388439 31054 raft_consensus.cc:2866] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:13.380851 30899 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:13.381098 31059 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:13.382637 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 957) took 1080 ms. Trace:
I20260809 07:33:13.390261 30899 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:13.389657 31172 rpcz_store.cc:276] 0809 07:33:12.302699 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:12.364007 (+ 61308us) service_pool.cc:227] Handling call
0809 07:33:12.389254 (+ 25247us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:12.389588 (+ 334us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:12.389617 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:12.390020 (+ 403us) log.cc:842] Serialized 26 byte log entry
0809 07:33:12.983403 (+593383us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:12.983447 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:12.983519 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:13.382454 (+398935us) raft_consensus.cc:1671] finished
0809 07:33:13.382474 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.382582 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6809,"thread_start_us":573557,"threads_started":1,"wal-append.queue_time_us":812919}
I20260809 07:33:13.391361 30899 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:13.349788 1327 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:13.395865 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47037.3us
I20260809 07:33:13.396646 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.365124 1384 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:13.359933 1417 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260809 07:33:13.346375 1246 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260809 07:33:13.364737 1361 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s user 0.008s sys 0.015s
I20260809 07:33:13.300558 1332 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260809 07:33:13.397948 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.392390 30899 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:13.398456 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e160777f9c924323b0223db59c1ae041"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:13.364127 1222 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.401857 30896 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:13.401327 854 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:13.251041 1392 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.373454 1183 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:13.429728 31120 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s user 0.021s sys 0.020s
I20260809 07:33:13.404433 1384 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:13.430068 31120 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.518s user 0.021s sys 0.020s
W20260809 07:33:13.410542 1361 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s user 0.008s sys 0.016s
I20260809 07:33:13.433549 30899 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260809 07:33:13.446799 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 568234us
I20260809 07:33:13.446798 30890 raft_consensus.cc:595] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.452610 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:13.432574 1391 consensus_meta.cc:287] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.502s user 0.008s sys 0.004s
I20260809 07:33:13.454610 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260809 07:33:13.460165 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260809 07:33:13.454993 1434 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260809 07:33:13.407200 1246 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:13.429281 854 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:13.410763 597 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:13.455416 1183 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.433123 1448 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:13.462579 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.469087 31049 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:13.466395 1267 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:13.474263 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:13.464166 1246 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.465662 854 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.467142 597 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:13.459996 1391 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.472689 1183 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:13.490820 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.481477 30890 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:13.498348 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.491974 597 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:13.511991 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:13.512799 31049 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:13.519120 1391 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.536520 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:13.482103 1267 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.509018 1281 consensus_queue.cc:1046] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260809 07:33:13.552647 31164 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260809 07:33:13.579111 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 182541us
I20260809 07:33:13.562855 31170 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:13.564392 31177 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:13.582492 30852 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.579667 31006 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:13.558887 31165 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:13.556361 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:13.543812 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 143357us
I20260809 07:33:13.543506 30890 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:13.563304 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 990) took 1171 ms. Trace:
I20260809 07:33:13.588613 30898 rpcz_store.cc:276] 0809 07:33:12.392061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:13.347228 (+955167us) service_pool.cc:227] Handling call
0809 07:33:13.347951 (+ 723us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:13.387987 (+ 40036us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.563073 (+175086us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b54003c2dd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:13.563233 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":71148288}
I20260809 07:33:13.593668 30853 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:13.593923 30852 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.594910 31175 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:13.590658 699 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:13.595661 699 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:13.599056 31006 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.598212 699 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:13.596055 1230 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260809 07:33:13.601761 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 992) took 1125 ms. Trace:
W20260809 07:33:13.603368 1466 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.155s user 0.000s sys 0.001s
I20260809 07:33:13.610452 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:13.562498 1298 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:13.609274 30906 rpcz_store.cc:276] 0809 07:33:12.476046 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:13.344630 (+868584us) service_pool.cc:227] Handling call
0809 07:33:13.344957 (+ 327us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:13.361723 (+ 16766us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:13.361755 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:13.362197 (+ 442us) log.cc:842] Serialized 26 byte log entry
0809 07:33:13.534624 (+172427us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:13.534659 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.565895 (+ 31236us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b5400281c54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:13.565942 (+ 47us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:13.601486 (+ 35544us) raft_consensus.cc:1671] finished
0809 07:33:13.601507 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.601704 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":37986560,"thread_start_us":138263,"threads_started":1,"wal-append.queue_time_us":224038}
I20260809 07:33:13.544466 1288 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:13.595709 30890 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:13.374022 30904 rpcz_store.cc:276] 0809 07:33:12.259606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:12.259810 (+ 204us) service_pool.cc:227] Handling call
0809 07:33:12.260114 (+ 304us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:12.293499 (+ 33385us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:12.293529 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:12.310657 (+ 17128us) log.cc:842] Serialized 52 byte log entry
0809 07:33:13.258231 (+947574us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:13.301911 (+ 43680us) log.cc:842] Serialized 14 byte log entry
0809 07:33:13.349117 (+ 47206us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.349193 (+ 76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:13.349233 (+ 40us) raft_consensus.cc:1671] finished
0809 07:33:13.349247 (+ 14us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.349366 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":138768,"prepare.queue_time_us":57192,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":11280,"raft.queue_time_us":37895,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":947163,"threads_started":1,"wal-append.queue_time_us":1027967}
I20260809 07:33:13.598238 1448 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.605408 1361 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 07:33:13.613276 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 127620us
I20260809 07:33:13.614214 31006 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.606745 1007 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.594864 1387 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260809 07:33:13.618223 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.608364 1365 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260809 07:33:13.631951 30852 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.613498 1468 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:13.641585 30896 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:13.614495 1310 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260809 07:33:13.613927 1288 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.636788 1361 raft_consensus.cc:686] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:13.616200 1298 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.636161 1493 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:13.683192 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 69048.1us
I20260809 07:33:13.667954 1365 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.657184 1468 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:13.656870 1504 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:13.678090 1014 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:13.683056 1310 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:13.682488 1493 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:13.689131 1281 raft_consensus.cc:3032] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
W20260809 07:33:13.701246 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 83119us
I20260809 07:33:13.697269 1507 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:13.687613 1468 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.695441 1497 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:13.699231 1472 log.cc:924] Time spent T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.140s user 0.000s sys 0.001s
I20260809 07:33:13.689215 1014 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:13.696112 1267 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:13.701126 1509 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:13.701459 1281 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:13.700878 1493 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:13.725725 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.701084 1365 consensus_queue.cc:1046] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260809 07:33:13.699211 1310 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.695623 1504 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.724226 699 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.725065 1267 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:13.722589 1509 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:13.738926 1298 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.706051 1507 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.731702 31182 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:13.741640 1504 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:13.705477 1361 consensus_queue.cc:237] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.744890 1267 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.724812 1497 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260809 07:33:13.742681 1387 raft_consensus.cc:3032] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:13.747500 1310 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.750644 1509 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.763530 1509 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:13.756399 31059 raft_consensus.cc:2396] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:13.759608 1387 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:13.744640 1014 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.752012 1281 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.770924 1014 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.754012 1267 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:13.771142 1497 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:13.748585 1384 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.772116 1497 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:13.772486 1497 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:13.756851 1246 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.773679 31057 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:13.771174 1281 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.751880 1327 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.773267 1497 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.755115 1507 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:13.751997 854 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.797430 1384 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.797914 1327 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.759402 1468 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.798979 854 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.799194 1246 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.758066 597 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.800089 1468 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.801311 597 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.801967 1331 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:13.752552 32198 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 4 LEARNER]: Advancing to term 5
I20260809 07:33:13.770855 1387 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.758765 1523 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.752498 1504 consensus_queue.cc:1046] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260809 07:33:13.797217 1497 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:13.803610 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:13.802408 1495 log.cc:924] Time spent T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Append to log took a long time: real 0.187s user 0.005s sys 0.000s
I20260809 07:33:13.745066 1521 consensus_queue.cc:1046] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260809 07:33:13.795096 30900 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:13.802443 1331 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:33:13.816473 1461 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.366s user 0.002s sys 0.000s
I20260809 07:33:13.818507 1267 raft_consensus.cc:2741] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 07:33:13.807245 1507 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260809 07:33:13.820164 30896 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:13.821413 30896 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260809 07:33:13.822736 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:13.815312 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.832535 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:13.834709 1267 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:13.835487 1267 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:13.839320 30896 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:13.841416 699 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.841037 1493 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.844858 1521 raft_consensus.cc:3032] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:13.845369 1521 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:13.846236 1521 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.847458 1521 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:13.854089 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.856418 1523 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
W20260809 07:33:13.866868 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 245787us
I20260809 07:33:13.867825 30853 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:13.870561 1214 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:13.871057 1214 raft_consensus.cc:686] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:13.872139 31005 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.871924 1214 consensus_queue.cc:237] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.873306 31005 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.874141 699 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.874711 699 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:13.875829 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 244590us
I20260809 07:33:13.875962 1083 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.876525 1281 raft_consensus.cc:3032] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:13.876914 1281 raft_consensus.cc:729] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:13.876988 30849 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 257003us
I20260809 07:33:13.878350 31005 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.879526 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.888054 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.888713 31005 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.890462 31005 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.890626 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.891666 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.892405 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.892745 1281 consensus_queue.cc:260] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:13.889575 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.895242 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:13.895845 31164 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:13.895912 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:13.897223 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 212023us
I20260809 07:33:13.897636 31005 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.901222 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.901329 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.902711 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.906999 31005 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:13.893756 1527 log.cc:924] Time spent T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260809 07:33:13.909503 30852 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.914491 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.888411 699 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:13.915658 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.895035 1438 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.924321 30852 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.894174 1506 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:13.899053 1523 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:13.895980 1246 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.960479 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.963562 32198 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 LEARNER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:13.900084 1384 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:13.923677 1517 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:13.969147 1529 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:13.969583 1529 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:13.970083 1475 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.475s user 0.003s sys 0.000s
I20260809 07:33:13.970458 1497 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:13.970877 1497 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:13.971313 1497 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.971928 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:13.973287 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:13.974161 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 995) took 1469 ms. Trace:
I20260809 07:33:13.957396 1439 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.896857 1007 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:13.974570 30911 rpcz_store.cc:276] 0809 07:33:12.504885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:13.345533 (+840648us) service_pool.cc:227] Handling call
0809 07:33:13.345828 (+ 295us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:13.392224 (+ 46396us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:13.392264 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:13.394531 (+ 2267us) log.cc:842] Serialized 26 byte log entry
0809 07:33:13.764635 (+370104us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b18002b4da0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411 00007f4094ecd5cf
0809 07:33:13.899082 (+134447us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:13.899114 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.899213 (+ 99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:13.973853 (+ 74640us) raft_consensus.cc:1671] finished
0809 07:33:13.973872 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.974099 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70672,"raft.queue_time_us":49243,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5648,"spinlock_wait_cycles":11518848,"thread_start_us":134284,"threads_started":1,"wal-append.queue_time_us":568590}
I20260809 07:33:13.908428 1298 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.970078 1529 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:13.980681 1507 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.981325 1507 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:13.924542 1508 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:13.906754 1208 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:13.927915 1281 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:13.986982 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 986) took 1663 ms. Trace:
I20260809 07:33:14.004439 30909 rpcz_store.cc:276] 0809 07:33:12.323226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:12.349021 (+ 25795us) service_pool.cc:227] Handling call
0809 07:33:12.349376 (+ 355us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:12.993445 (+644069us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:12.993478 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:13.005704 (+ 12226us) log.cc:842] Serialized 52 byte log entry
0809 07:33:13.360245 (+354541us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:13.544070 (+183825us) log.cc:842] Serialized 14 byte log entry
0809 07:33:13.544314 (+ 244us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:13.544401 (+ 87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:13.955564 (+411163us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:13.975711 (+ 20147us) raft_consensus.cc:1671] finished
0809 07:33:13.986789 (+ 11078us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:13.986923 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":57408,"prepare.queue_time_us":22807,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":72409,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":6220,"spinlock_wait_cycles":4514560,"thread_start_us":1052234,"threads_started":3,"wal-append.queue_time_us":439642}
I20260809 07:33:13.909206 1183 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.909351 1370 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.959014 1523 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:13.975649 1438 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:13.906666 1310 consensus_queue.cc:1046] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260809 07:33:13.982503 1504 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:33:13.974773 1332 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:13.969395 1384 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:14.013417 1331 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.014147 1493 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:13.997504 1246 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:14.019757 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:14.031303 30906 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:14.019098 30898 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:14.023909 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:14.025755 30911 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:14.028378 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:14.032615 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:14.032070 30904 raft_consensus.cc:2354] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:14.033078 30899 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:14.037433 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 123021us
I20260809 07:33:14.008030 1523 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:13.924639 1457 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260809 07:33:14.062883 31005 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:13.914011 1515 consensus_queue.cc:1046] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.124s
I20260809 07:33:13.987596 699 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260809 07:33:14.062244 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:14.090204 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:14.105804 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 132584us
I20260809 07:33:14.094659 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:14.108511 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:14.109459 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 73116.8us
W20260809 07:33:14.113108 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 205143us
I20260809 07:33:14.037256 1504 raft_consensus.cc:686] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.107877 31164 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:14.115348 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:13.960358 1506 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:14.120208 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:14.122036 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 58830.1us
I20260809 07:33:14.124679 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.125362 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:14.125591 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.127010 31045 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:14.127635 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.126967 31057 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:14.127092 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.129200 31055 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.128358 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.116230 1384 raft_consensus.cc:3032] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:14.129802 31045 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.091044 1083 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260809 07:33:14.116256 1521 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:14.059968 1455 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260809 07:33:14.115869 1310 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:14.138257 31055 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:14.139592 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.118496 1457 raft_consensus.cc:3032] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:14.146747 31045 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:14.147140 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.101418 1497 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.123046 1508 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:14.150421 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:14.166049 31046 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:14.169565 31049 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.172448 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:14.172672 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:14.098212 1529 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.173849 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:14.090371 1246 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.174605 31058 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:14.173359 1545 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260809 07:33:14.146821 1515 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260809 07:33:14.146643 1298 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:14.178399 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:14.189514 31169 raft_consensus.cc:2354] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:14.182173 31182 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:14.185050 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:14.192880 31182 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 07:33:14.135133 1384 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:14.105187 1523 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:14.266528 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 141267us
I20260809 07:33:14.272150 31049 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:14.166718 1457 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:14.274605 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:14.183792 1347 raft_consensus.cc:979] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: : Instructing follower a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:14.167616 699 consensus_queue.cc:1046] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
W20260809 07:33:14.177946 1535 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20260809 07:33:14.295338 31181 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:14.198587 1298 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:14.174785 1267 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:14.175799 1521 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:14.282254 31120 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.303680 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.288870 31006 leader_election.cc:398] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:14.307087 31006 leader_election.cc:401] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:14.181466 1387 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:14.177745 1506 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:14.310942 1506 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:14.311422 1506 raft_consensus.cc:686] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.312317 1506 consensus_queue.cc:237] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.184543 1331 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.282222 1529 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:14.176211 1183 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
I20260809 07:33:14.341410 31059 raft_consensus.cc:2396] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:14.299386 1298 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.343331 1246 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.186888 1542 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.131s
I20260809 07:33:14.263288 1438 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260809 07:33:14.351850 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 165336us
I20260809 07:33:14.317323 1267 raft_consensus.cc:686] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.346896 1545 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:14.320326 1521 raft_consensus.cc:686] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.376655 31005 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.296474 1507 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260809 07:33:14.177191 1504 consensus_queue.cc:237] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.349066 1007 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:14.317296 1014 consensus_queue.cc:1046] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260809 07:33:14.421039 30852 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.421882 31049 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260809 07:33:14.422531 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 290558us
I20260809 07:33:14.392256 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.295353 1540 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260809 07:33:14.396661 1365 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:14.403029 1507 raft_consensus.cc:3032] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:14.386794 1246 consensus_queue.cc:1046] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260809 07:33:14.404769 1523 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:14.385672 1545 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:14.405561 1497 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:14.410728 1521 consensus_queue.cc:237] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.351936 1183 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260809 07:33:14.350551 1457 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.417538 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 807) took 1188 ms. Trace:
I20260809 07:33:14.447116 30890 rpcz_store.cc:276] 0809 07:33:13.228920 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:13.616007 (+387087us) service_pool.cc:227] Handling call
0809 07:33:13.631644 (+ 15637us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:13.874882 (+243238us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:13.874914 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:13.875401 (+ 487us) log.cc:842] Serialized 26 byte log entry
0809 07:33:14.114334 (+238933us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:14.114613 (+ 279us) log.cc:842] Serialized 14 byte log entry
0809 07:33:14.416923 (+302310us) spinlock_profiling.cc:243] Waited 962 us on lock 0x7b18002e5a00. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c990 00007f4091fce864 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7 00007f4094f14058
0809 07:33:14.417127 (+ 204us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:14.417205 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:14.417248 (+ 43us) raft_consensus.cc:1671] finished
0809 07:33:14.417339 (+ 91us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:14.417477 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":267832,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":3080,"raft.queue_time_us":547302,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":166,"spinlock_wait_cycles":2213120,"thread_start_us":238626,"threads_started":1,"wal-append.queue_time_us":259921}
I20260809 07:33:14.419736 30930 tablet_copy_service.cc:342] P 2c9de56c3fa3487c9dac990675c34be5: Request end of tablet copy session 28b9965e21374d26b0ac2014c9bafdff-335fccb907184174b4a6428ce262e7dc received from {username='slave'} at 127.0.0.1:54234
I20260809 07:33:14.395099 1438 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:14.423898 1468 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:14.424178 1508 raft_consensus.cc:686] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:14.437881 1545 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:14.440624 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48464.3us
W20260809 07:33:14.442803 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 126107us
I20260809 07:33:14.422250 823 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:14.431663 1391 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:14.432143 1507 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.443482 1007 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:14.443960 1540 consensus_queue.cc:1225] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 2c9de56c3fa3487c9dac990675c34be5 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260809 07:33:14.439261 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 135679us
W20260809 07:33:14.471912 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 46192.8us
I20260809 07:33:14.454520 30930 tablet_copy_service.cc:433] P 2c9de56c3fa3487c9dac990675c34be5: ending tablet copy session 28b9965e21374d26b0ac2014c9bafdff-335fccb907184174b4a6428ce262e7dc on tablet 335fccb907184174b4a6428ce262e7dc with peer 28b9965e21374d26b0ac2014c9bafdff
I20260809 07:33:14.484815 31005 leader_election.cc:398] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:14.478816 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.455497 1332 raft_consensus.cc:1067] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Signalling peer a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:14.438478 1214 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260809 07:33:14.492774 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.467357 1457 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:14.498906 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.497354 30852 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:14.501101 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.461983 1468 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:14.501461 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 51003.4us
I20260809 07:33:14.530707 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:14.533556 31169 raft_consensus.cc:2396] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:14.508615 31182 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:14.446506 1267 consensus_queue.cc:237] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.471846 823 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.543550 1578 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:14.532924 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.505010 1504 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:14.557561 1504 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.559324 1542 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:14.559825 1542 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.560388 1492 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:14.563190 1331 raft_consensus.cc:3032] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:14.563593 1331 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:14.544657 1208 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:14.564177 1288 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.564325 1331 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.565380 1331 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:14.567795 1578 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.504357 1508 consensus_queue.cc:237] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.569792 1578 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.572014 1504 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:14.574461 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.575803 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.506361 1497 raft_consensus.cc:2741] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260809 07:33:14.556835 1574 log.cc:924] Time spent T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20260809 07:33:14.502882 1562 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:14.533857 1506 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:14.580592 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.505707 1566 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:14.547063 30853 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:14.545576 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 48323.3us
I20260809 07:33:14.564265 1208 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.588779 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.596521 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:14.573432 1559 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:14.559769 1468 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.576531 1281 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:14.623781 1569 log.cc:924] Time spent T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260809 07:33:14.623471 1559 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.625735 1559 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.628011 1281 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:14.628456 1281 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.634020 1545 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:14.635476 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 88494.3us
I20260809 07:33:14.636650 30852 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:14.637334 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48631.6us
I20260809 07:33:14.580541 1523 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:14.658365 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.660037 31005 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.661471 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.580690 1492 raft_consensus.cc:686] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:14.663050 31005 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.663654 31172 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:14.684966 31172 pending_rounds.cc:76] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 2
I20260809 07:33:14.685206 31172 pending_rounds.cc:97] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260809 07:33:14.685434 31172 raft_consensus.cc:2866] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:14.586740 1562 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:14.668668 1288 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:14.670598 31182 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:14.581538 1549 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:14.668116 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:14.687526 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:14.589845 1497 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:14.533491 1507 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.692241 30890 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:14.671716 1246 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:14.691675 1504 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:14.709739 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 70069.3us
I20260809 07:33:14.659227 1468 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:14.662494 854 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:14.711618 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:14.710883 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:14.897842 30898 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:14.704934 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:14.716809 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:14.899945 30900 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:14.669795 1438 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:14.689533 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:14.688318 1310 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:14.728869 31045 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e160777f9c924323b0223db59c1ae041"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:36492
I20260809 07:33:14.913014 31045 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:33:14.913427 31045 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:14.731367 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:14.731794 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:14.694280 1562 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.714195 1589 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:14.753919 1549 raft_consensus.cc:686] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.753803 1504 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:14.821693 1535 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
W20260809 07:33:14.834827 1593 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260809 07:33:14.838995 1545 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:14.846391 1529 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:14.848867 31164 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:14.853820 31181 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:14.856750 1384 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.718379 1214 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.106s
I20260809 07:33:14.693667 1365 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:14.916188 31057 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:14.916268 31051 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:14.925196 31045 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.927357 1504 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:14.913810 1507 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:14.860689 1298 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.826128 1492 consensus_queue.cc:237] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.867158 823 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.941861 31045 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.929068 597 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:14.926656 1392 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:14.944855 31164 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:14.942685 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:14.912811 1281 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:14.936179 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 246507us
W20260809 07:33:14.941316 1601 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.083s user 0.001s sys 0.000s
I20260809 07:33:14.922008 1332 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
W20260809 07:33:14.956653 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 202209us
I20260809 07:33:14.937960 1468 raft_consensus.cc:686] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:14.943938 1384 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:14.934067 1288 consensus_queue.cc:1046] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260809 07:33:14.935827 1589 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:14.976090 31048 raft_consensus.cc:1261] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260809 07:33:14.985860 31048 pending_rounds.cc:76] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:14.986135 31048 pending_rounds.cc:97] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260809 07:33:14.986380 31048 raft_consensus.cc:2866] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260809 07:33:14.978189 1603 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.058s user 0.001s sys 0.000s
I20260809 07:33:14.969974 597 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.989473 597 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.988257 1589 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.967271 1549 consensus_queue.cc:237] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:14.972460 1542 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.969106 1534 raft_consensus.cc:3032] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:14.982151 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:14.956287 1566 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:14.954407 1195 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260809 07:33:14.957230 1523 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.002755 31005 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:14.943758 1327 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260809 07:33:14.958431 802 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260809 07:33:14.981815 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:15.023705 1566 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.024120 1623 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:15.031338 1281 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.034715 1468 consensus_queue.cc:237] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.001895 1534 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.040549 1208 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.040997 1534 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.006682 1310 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:15.042562 1310 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:15.042961 1310 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:15.044395 1298 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:14.999579 1545 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260809 07:33:14.979111 1392 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.000743 1457 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260809 07:33:15.047372 1612 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.073s user 0.002s sys 0.000s
I20260809 07:33:15.048189 1392 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.004926 1542 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:15.050573 1542 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.052663 1542 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.060468 31164 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:15.063380 1515 raft_consensus.cc:3032] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.067862 1523 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.071152 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:15.072690 31177 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.042201 1534 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.035552 1509 raft_consensus.cc:1050] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
I20260809 07:33:15.075019 31006 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:15.063834 1515 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:15.099032 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 155008us
W20260809 07:33:15.095432 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 113701us
I20260809 07:33:15.095707 1246 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:15.138785 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 43943.7us
I20260809 07:33:15.094630 1298 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:15.098026 1208 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.138353 1613 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:15.154009 1331 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:14.973141 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:15.172201 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:15.171955 1504 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.138460 1562 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.184661 1438 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260809 07:33:15.187325 1298 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:15.183626 1331 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.187776 1298 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.188687 1298 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.189394 1331 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.193898 1613 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.194864 1613 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.196053 1504 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.197827 31169 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:15.198900 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.199515 31169 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:15.200626 31165 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
W20260809 07:33:15.204802 1618 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.132s user 0.000s sys 0.000s
I20260809 07:33:15.205015 1298 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.158102 1246 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.207682 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.208415 31046 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:15.208619 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.208768 1246 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.209177 31048 raft_consensus.cc:2362] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5. Re-sending same reply.
I20260809 07:33:15.210083 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.210837 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.211421 31055 raft_consensus.cc:2362] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 3. Re-sending same reply.
I20260809 07:33:15.187693 1562 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.212338 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:15.213291 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.212206 31177 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:15.210794 31045 raft_consensus.cc:2354] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6.
I20260809 07:33:15.215121 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.196272 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.218962 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.219254 1246 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:15.219745 1370 raft_consensus.cc:979] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: : Instructing follower a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:15.220309 1365 raft_consensus.cc:3032] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.220691 1365 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.220345 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.229215 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.215564 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.216230 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.231362 31045 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:15.233135 31164 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:15.240269 31055 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260809 07:33:15.240547 31048 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260809 07:33:15.241004 31165 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:15.243840 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.247040 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.253700 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.254717 31177 raft_consensus.cc:1261] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:15.256538 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:15.201761 1310 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.269574 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.260064 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.210477 1509 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:15.238734 1014 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.232635 1578 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.263904 31120 leader_election.cc:398] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6.
I20260809 07:33:15.218870 1384 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260809 07:33:15.253603 1545 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.280993 31057 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260809 07:33:15.173264 1515 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.284127 31048 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:15.285857 31172 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:15.273793 1544 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.270251 1281 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.293095 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.296700 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.299254 31006 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.307682 31006 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:15.315215 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.315510 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.238976 1365 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.323458 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.326771 1497 consensus_queue.cc:1046] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:15.327715 1497 consensus_queue.cc:1046] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:15.328548 1497 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.328860 1497 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.318279 1515 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.330962 1497 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.331960 1506 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:15.282013 1014 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.300576 1281 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.348543 1281 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.348747 31048 raft_consensus.cc:1261] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:15.343926 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.315191 1384 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:15.344367 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.349750 1549 raft_consensus.cc:3032] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:15.348497 1014 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.345324 31182 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:15.329973 1523 consensus_queue.cc:1046] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260809 07:33:15.285048 1545 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.349979 31055 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:15.283743 1578 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.355604 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.349957 31048 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:15.322949 1568 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:15.356261 31182 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:15.319147 1183 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260809 07:33:15.361253 31006 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.364805 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.351058 1310 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.365516 31048 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:15.362041 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 23) took 2259 ms. Trace:
I20260809 07:33:15.351923 1549 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.367869 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:15.372318 1549 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.374584 1549 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.371680 30744 rpcz_store.cc:276] 0809 07:33:13.102514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:13.103220 (+ 706us) service_pool.cc:227] Handling call
0809 07:33:15.361982 (+2258762us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:13.598013 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:13.671799 (+ 73786us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:13.673531 (+ 1732us) write_op.cc:435] Acquired schema lock
0809 07:33:13.673557 (+ 26us) tablet.cc:660] Decoding operations
0809 07:33:13.698024 (+ 24467us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:13.699966 (+ 1942us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:13.700003 (+ 37us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:33:13.700246 (+ 243us) tablet.cc:699] Row locks acquired
0809 07:33:13.700273 (+ 27us) write_op.cc:260] PREPARE: finished
0809 07:33:13.700476 (+ 203us) write_op.cc:270] Start()
0809 07:33:13.703304 (+ 2828us) write_op.cc:276] Timestamp: P: 1786260793700425 usec, L: 0
0809 07:33:13.704643 (+ 1339us) op_driver.cc:331] REPLICATION: starting
0809 07:33:13.736497 (+ 31854us) log.cc:842] Serialized 556 byte log entry
0809 07:33:14.333975 (+597478us) op_driver.cc:444] REPLICATION: finished
0809 07:33:14.555284 (+221309us) write_op.cc:301] APPLY: starting
0809 07:33:14.572607 (+ 17323us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:14.582634 (+ 10027us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:14.635935 (+ 53301us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:14.715350 (+ 79415us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:14.812548 (+ 97198us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:33:14.820794 (+ 8246us) write_op.cc:312] APPLY: finished
0809 07:33:14.837365 (+ 16571us) log.cc:842] Serialized 22 byte log entry
0809 07:33:14.837571 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:14.837659 (+ 88us) write_op.cc:454] Released schema lock
0809 07:33:14.837699 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:14.837942 (+ 243us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:14.837968 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":332227,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":181373,"apply.run_cpu_time_us":2517,"apply.run_wall_time_us":303502,"num_ops":1,"prepare.queue_time_us":456680,"prepare.run_cpu_time_us":3668,"prepare.run_wall_time_us":148598,"replication_time_us":613843,"thread_start_us":127353,"threads_started":1}]]}
I20260809 07:33:15.381366 1578 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.384336 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.362233 1365 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.362491 1497 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.301723 1544 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.363853 1568 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.400923 1545 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.406922 1544 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.369807 1387 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:15.411618 1387 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:15.411973 1387 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.415599 31170 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260809 07:33:15.360661 1384 consensus_queue.cc:1046] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260809 07:33:15.388772 1529 consensus_queue.cc:1046] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260809 07:33:15.403944 1504 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:15.421191 1637 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
I20260809 07:33:15.390619 1523 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260809 07:33:15.481606 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.482429 31172 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:15.397965 1267 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260809 07:33:15.403456 1562 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260809 07:33:15.419203 1607 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:15.485590 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 100388us
W20260809 07:33:15.493342 1647 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
W20260809 07:33:15.494834 1652 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260809 07:33:15.495950 1387 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.477851 1504 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:15.498281 1387 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.498588 1504 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.479563 1183 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.500765 31006 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.506809 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:15.507400 31177 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.508296 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.508524 31057 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:15.509140 31045 raft_consensus.cc:2396] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:15.511039 31058 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:15.510008 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.513607 31059 raft_consensus.cc:2396] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:15.517874 31177 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:15.400942 1568 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.519985 1515 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:15.520224 1544 consensus_queue.cc:1046] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:15.521416 1517 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:15.504774 1183 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.523666 1183 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.526433 1515 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:15.526818 1515 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.530683 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.535530 31006 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.539577 31051 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:15.539847 31058 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260809 07:33:15.540256 31058 pending_rounds.cc:76] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 3
I20260809 07:33:15.540482 31058 pending_rounds.cc:97] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260809 07:33:15.540715 31058 raft_consensus.cc:2866] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:15.541420 1281 raft_consensus.cc:3032] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.553581 31181 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:15.543237 1281 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.553310 1662 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.553418 1384 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.557235 1534 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:15.565841 1662 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.586802 31058 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:15.584265 1534 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.606848 1606 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:15.607385 32198 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 LEARNER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 07:33:15.607692 31055 raft_consensus.cc:1261] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260809 07:33:15.608304 31055 pending_rounds.cc:76] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 3
I20260809 07:33:15.608520 31055 pending_rounds.cc:97] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260809 07:33:15.608729 31055 raft_consensus.cc:2866] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:15.570456 1384 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.617772 31058 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260809 07:33:15.619345 1515 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.621340 1515 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.622462 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:15.623041 31169 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.631289 31169 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:15.632323 31006 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.633437 1578 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:15.633888 1578 raft_consensus.cc:686] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.595237 1534 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.634717 1578 consensus_queue.cc:237] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.643216 1540 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:15.643739 1540 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.645666 1540 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.646759 699 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.647169 699 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.647921 31057 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:15.648044 699 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.649308 699 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.649643 1613 raft_consensus.cc:3032] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:15.650002 1613 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.650903 1568 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:15.651429 1568 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.653396 1568 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.654120 1542 raft_consensus.cc:3032] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.654502 1542 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:15.655251 1542 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.662608 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.665053 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.577021 1562 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260809 07:33:15.616676 1298 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:15.574697 1281 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.682296 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.686290 597 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:15.686837 597 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.687834 1347 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.688308 1347 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.690459 1494 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.690907 1494 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.691433 1663 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:15.691819 1507 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.691908 1663 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.692354 1507 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.692838 1663 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.694998 1373 raft_consensus.cc:1050] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5: attempting to promote NON_VOTER 28b9965e21374d26b0ac2014c9bafdff to VOTER
I20260809 07:33:15.665488 1542 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.635452 1606 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.060s
I20260809 07:33:15.705425 31054 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:15.642534 1438 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260809 07:33:15.690143 1347 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.680442 1331 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:15.669719 1607 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260809 07:33:15.697409 1298 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.698252 1515 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.722671 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:15.727147 1670 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
I20260809 07:33:15.727582 1613 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.694255 1507 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.682673 1566 consensus_queue.cc:1046] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260809 07:33:15.699640 1568 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:15.695869 1007 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:15.760119 1515 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:15.763863 30849 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 85490.4us
I20260809 07:33:15.769670 1298 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.754688 1365 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.755823 1370 raft_consensus.cc:1067] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Signalling peer a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:15.753605 1208 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.774748 31170 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 07:33:15.775354 31051 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:15.783146 31051 pending_rounds.cc:76] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:15.783403 31051 pending_rounds.cc:97] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 07:33:15.783620 31051 raft_consensus.cc:2866] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:15.784487 31058 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:15.784647 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.775157 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.778798 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.787070 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:15.771878 1613 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:15.761008 1331 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.798653 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.799338 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.778656 1007 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.779307 1568 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.800506 1662 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:15.782271 1365 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:15.818821 1680 log.cc:924] Time spent T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.109s user 0.001s sys 0.000s
I20260809 07:33:15.797260 1298 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260809 07:33:15.788331 1208 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.805178 1384 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.834043 31051 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:15.807765 597 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.844370 31057 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:15.845562 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.846148 30904 raft_consensus.cc:3032] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:15.834353 1621 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:15.847043 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.847922 30900 raft_consensus.cc:2396] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 07:33:15.848278 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.836611 1515 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.838174 1007 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.839896 1208 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.841393 1365 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.846952 30896 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:15.846464 30904 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:15.857611 30899 raft_consensus.cc:1261] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:15.851359 30909 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260809 07:33:15.846463 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.859804 30906 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.852936 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.853753 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.828230 1494 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.854076 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.855067 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.860814 30911 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:15.860519 30891 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:15.861521 30890 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:15.864643 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.865186 30890 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.877455 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.860569 30900 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:15.846953 1621 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.879382 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.882817 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.883338 30891 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:15.884013 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.884620 30891 raft_consensus.cc:2396] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:15.884439 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.885708 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.875247 30904 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.886698 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.887885 30904 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.888343 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.888597 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.889889 30891 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 07:33:15.887553 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.896404 30904 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:15.898048 30890 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:15.898901 30900 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:15.904343 30911 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:15.905094 597 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:15.910684 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:15.912945 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.913838 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.914698 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.915747 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.915904 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.916021 30906 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:15.916270 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.917584 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.841539 1373 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 3, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:15.889994 1288 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:15.918675 30896 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:15.937016 597 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:15.917409 1208 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:15.937469 597 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:15.937798 1208 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:15.939747 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 230644us
I20260809 07:33:15.939873 1208 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.937834 597 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.944693 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:15.945282 31045 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:15.947077 1544 raft_consensus.cc:3032] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:15.947472 1544 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:15.948303 1544 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:15.949424 1544 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:15.949546 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.950963 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.951552 31169 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:15.951745 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.952849 31172 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:15.950556 1281 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:15.954452 1568 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.955149 31006 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.955533 1331 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:15.956063 1331 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:15.956383 1331 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:15.956543 1681 log.cc:924] Time spent T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.177s user 0.000s sys 0.000s
I20260809 07:33:15.957059 1331 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.957417 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:15.970139 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.923564 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.972338 30904 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:15.974509 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.974517 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:15.971123 31174 raft_consensus.cc:2396] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:15.975154 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:15.929065 30891 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:15.977906 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:15.978632 31170 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:15.980336 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:15.987709 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 45457.3us
I20260809 07:33:16.032778 30909 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:16.041215 31006 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:16.042268 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53838.3us
I20260809 07:33:16.044754 31170 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:16.042943 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:16.045672 30896 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:16.052016 30904 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:16.053380 30904 raft_consensus.cc:1261] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 07:33:16.054692 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:16.059136 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.059826 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.060020 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.061088 1607 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:16.062088 31170 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:16.063169 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.064680 31170 raft_consensus.cc:1261] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:15.970278 1509 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:16.042855 1507 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:16.042927 1568 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:16.061627 1607 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:16.069299 30904 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260809 07:33:16.068534 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.071847 31005 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.109023 30852 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.090298 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.081900 31170 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:16.044942 1007 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:16.114861 31057 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:16.042131 1288 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.070875 1521 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:16.108035 1566 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:16.142644 31169 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:16.147302 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.053803 1448 consensus_queue.cc:1046] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260809 07:33:16.120726 31177 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:16.068151 1559 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.140298 30852 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.070441 1195 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.113514 1507 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:16.164968 30852 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.123559 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.117141 1007 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:16.151298 1449 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 07:33:16.163887 1504 consensus_meta.cc:287] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.665s user 0.010s sys 0.003s
I20260809 07:33:16.110287 1509 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:16.118412 1568 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:16.167047 1679 log.cc:924] Time spent T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.523s user 0.001s sys 0.004s
I20260809 07:33:16.172909 30904 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
W20260809 07:33:16.174113 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 54142.9us
I20260809 07:33:16.176636 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:16.167171 1507 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:16.167586 802 raft_consensus.cc:1013] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: attempt to promote peer 28b9965e21374d26b0ac2014c9bafdff: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260809 07:33:16.168610 1007 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:16.170491 1214 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:16.174896 1685 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:16.155123 1517 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260809 07:33:16.150229 1566 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.167718 1195 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.194268 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:16.183192 1214 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:16.171159 1455 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260809 07:33:16.168198 1347 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260809 07:33:16.196532 30906 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:16.197840 1509 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.198849 1521 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:16.186185 1685 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:16.214596 1685 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:16.215046 1288 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:16.199239 1695 log.cc:924] Time spent T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.084s user 0.001s sys 0.000s
I20260809 07:33:16.200745 1566 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:16.204737 30852 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.202271 1214 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:16.174914 1504 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.212369 597 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.217569 30852 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.214061 30906 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:16.229522 597 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.231442 1222 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:16.231947 1439 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:16.238716 1578 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:16.238901 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 167145us
I20260809 07:33:16.241042 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.243269 1391 raft_consensus.cc:3032] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:16.240991 1310 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:16.243418 1696 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 07:33:16.243862 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 104579us
I20260809 07:33:16.245290 1298 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:16.245586 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.245759 1298 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.246215 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.246410 1696 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:16.246842 1696 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:16.247587 1578 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:16.248167 1578 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:16.248514 1578 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:16.249642 1614 raft_consensus.cc:3032] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:16.247767 1298 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.252761 31059 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260809 07:33:16.216141 1521 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:16.254230 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.243700 1391 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:16.256687 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:16.257941 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 77044.8us
I20260809 07:33:16.258700 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.259601 31059 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:16.261752 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.250012 1614 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:16.173957 1607 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:16.275003 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 57506.7us
I20260809 07:33:16.275442 1222 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:16.261749 1014 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:16.275727 30852 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.281441 31177 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:16.281917 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.282485 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:16.283597 31170 raft_consensus.cc:1261] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260809 07:33:16.211154 1370 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260809 07:33:16.231640 1496 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260809 07:33:16.292335 30852 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.299400 1504 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:16.293447 1391 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.293244 1623 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
I20260809 07:33:16.301112 1391 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:16.302981 1494 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:16.263280 1347 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260809 07:33:16.307971 1370 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:16.308429 1370 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.312233 1331 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.313149 31181 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:16.313751 1686 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:16.314518 32198 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 LEARNER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:16.296152 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.316551 1686 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:16.291103 1568 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.317243 1686 consensus_queue.cc:1225] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:16.319335 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.320114 31164 raft_consensus.cc:2379] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:16.321292 1370 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.326035 30909 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260809 07:33:16.327069 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:16.327431 1497 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.327622 30909 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:16.327909 1497 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.328758 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.330178 30899 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:16.353251 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.355369 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.355273 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.356375 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.357487 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.358970 30891 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:16.360328 30890 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:16.360577 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.361212 30899 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:16.362919 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:16.364113 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.365317 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.366035 30899 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:16.366928 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.368829 30899 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:16.370819 30890 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260809 07:33:16.374101 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 78042.4us
I20260809 07:33:16.374729 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.378362 31059 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:36492
I20260809 07:33:16.378805 31059 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:33:16.379011 31059 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:16.350431 1014 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:16.384527 30891 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:16.360816 1559 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:16.390368 31051 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:16.375521 1347 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.296267 1435 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.390180 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.391211 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:16.388612 30827 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet ee4aa83021af4237806656803bff3e10, TS 2c9de56c3fa3487c9dac990675c34be5 -> TS 28b9965e21374d26b0ac2014c9bafdff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.314199 1331 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.360051 1281 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 07:33:16.396806 30827 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet ee4aa83021af4237806656803bff3e10, TS 2c9de56c3fa3487c9dac990675c34be5 -> TS 28b9965e21374d26b0ac2014c9bafdff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.363355 699 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.377887 1504 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:16.397010 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.404518 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:16.398303 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 125997us
I20260809 07:33:16.394865 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.291254 1438 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260809 07:33:16.383991 1568 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:16.402103 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.412950 31164 raft_consensus.cc:2379] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:16.385103 1523 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.407042 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.408630 31059 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.409379 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.349576 1696 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.295531 1492 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260809 07:33:16.414047 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:16.397431 1549 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:16.452934 30852 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.454197 30852 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.427821 1507 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.405251 1195 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.461771 31169 raft_consensus.cc:2379] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:16.411839 1529 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 07:33:16.395884 1566 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.416829 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:16.470425 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 53701us
I20260809 07:33:16.471140 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.424804 31059 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.426008 31177 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:16.476220 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:16.425678 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.426558 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.477485 31048 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:16.415093 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.479009 31051 raft_consensus.cc:2379] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:16.430612 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:16.479772 31170 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:16.436122 1214 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.437738 1568 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.438259 1331 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.293022 1614 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.442107 1310 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:16.446090 1007 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.447618 1521 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:16.449110 1710 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.153s user 0.000s sys 0.000s
W20260809 07:33:16.449770 1709 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.159s user 0.000s sys 0.000s
I20260809 07:33:16.450902 1455 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:16.402908 1014 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:16.460798 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:16.524160 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 63435.6us
I20260809 07:33:16.525302 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:16.387364 1517 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:16.463719 31172 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:16.461850 1496 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.465252 1392 consensus_queue.cc:1046] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:16.461625 1504 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:16.465999 1507 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:16.466687 1497 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:16.414775 699 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.543824 31174 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:16.467571 1696 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.400768 1347 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.410440 30909 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:16.478057 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.461774 1435 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:16.502635 30904 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:16.505679 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.506768 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:16.476861 1529 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260809 07:33:16.510438 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 135758us
I20260809 07:33:16.555699 31049 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:16.513140 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:16.559513 31048 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:16.532327 1492 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:16.491019 1566 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.529165 30852 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.495363 1685 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.540594 1578 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.541538 1720 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.544477 31174 raft_consensus.cc:595] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.546669 31045 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260809 07:33:16.550407 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 72447us
I20260809 07:33:16.574594 1566 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:16.575814 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.495739 1568 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.535487 1288 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:16.558907 31005 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.511896 31170 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:16.499389 1331 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.553807 31051 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:16.529206 31182 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:16.565308 30852 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.565915 699 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.567107 1439 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:16.613246 31049 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:16.554082 1507 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260809 07:33:16.611433 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52614.3us
I20260809 07:33:16.611460 31006 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:16.609418 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.619190 31059 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260809 07:33:16.619560 31059 pending_rounds.cc:76] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 3
I20260809 07:33:16.619777 31059 pending_rounds.cc:97] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260809 07:33:16.619997 31059 raft_consensus.cc:2866] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:16.621172 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:16.618767 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53548.9us
I20260809 07:33:16.547561 1504 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.626461 30852 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.568090 1492 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.629860 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.700243 31048 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:16.687371 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 56713.9us
I20260809 07:33:16.707096 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:16.686851 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 57225.2us
W20260809 07:33:16.700790 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 66777.2us
I20260809 07:33:16.612617 1288 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:16.576601 1566 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260809 07:33:16.686169 699 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
W20260809 07:33:16.623749 1534 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s user 0.009s sys 0.012s
I20260809 07:33:16.576668 1281 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
I20260809 07:33:16.617707 1717 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 07:33:16.720104 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 90310.5us
I20260809 07:33:16.689136 1222 raft_consensus.cc:2741] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 07:33:16.473313 32198 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:16.625110 1195 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:16.624394 1705 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:16.768965 31174 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:16.623026 1707 consensus_queue.cc:1046] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.137s
W20260809 07:33:16.753784 1534 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.113s user 0.009s sys 0.012s
I20260809 07:33:16.631116 1507 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.138s
I20260809 07:33:16.629420 1523 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.130s
I20260809 07:33:16.629572 1544 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:16.717646 1288 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:16.799835 1521 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:16.803388 1504 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:16.805186 1685 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.805989 1568 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:16.806545 1683 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260809 07:33:16.806288 1689 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:16.807502 1578 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.697796 1493 raft_consensus.cc:3032] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:16.701664 1606 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260809 07:33:16.764415 1332 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:16.614988 1720 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:16.895084 1700 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.833s user 0.005s sys 0.000s
I20260809 07:33:16.896745 1720 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.718207 1492 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.840152 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.792171 1281 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.791929 1717 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:16.720851 1435 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.773094 1517 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260809 07:33:16.808323 1439 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:16.852569 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:16.701831 1696 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
W20260809 07:33:16.864912 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 157886us
I20260809 07:33:16.879688 1332 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:16.879930 1347 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.838565 1007 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.841534 1521 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:16.924088 1718 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260809 07:33:16.865303 1578 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.882417 1706 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.195s
I20260809 07:33:16.863209 1493 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:16.891846 31057 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:16.927348 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:16.924913 1717 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:16.931553 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.932958 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:16.854672 1712 log.cc:924] Time spent T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.141s user 0.003s sys 0.003s
I20260809 07:33:16.932512 1439 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } }
I20260809 07:33:16.923336 1281 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.938053 1288 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.938194 1521 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.922807 1720 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.948451 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1066) took 1007 ms. Trace:
I20260809 07:33:16.948468 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:16.948828 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:16.942127 1492 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.932711 1332 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:16.946821 1717 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:16.952946 31054 rpcz_store.cc:276] 0809 07:33:15.941047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:15.941918 (+ 871us) service_pool.cc:227] Handling call
0809 07:33:15.942251 (+ 333us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:15.942887 (+ 636us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:15.943163 (+ 276us) log.cc:842] Serialized 14 byte log entry
0809 07:33:16.055440 (+112277us) log.cc:842] Serialized 14 byte log entry
0809 07:33:16.233447 (+178007us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b18000f3580. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c990 00007f4091fce864 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7 00007f4094f14058
0809 07:33:16.233597 (+ 150us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:16.234026 (+ 429us) log.cc:842] Serialized 26 byte log entry
0809 07:33:16.234140 (+ 114us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:16.234167 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:16.234243 (+ 76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:16.504693 (+270450us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b18000f3580. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511
0809 07:33:16.894621 (+389928us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:16.948277 (+ 53656us) raft_consensus.cc:1671] finished
0809 07:33:16.948298 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:16.948402 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":162926,"prepare.queue_time_us":262,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3538,"raft.queue_time_us":183846,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":13232,"spinlock_wait_cycles":79870976,"thread_start_us":111717,"threads_started":1,"wal-append.queue_time_us":118187}
I20260809 07:33:16.930637 1435 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.943120 1014 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:16.946648 1578 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.964394 31005 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:16.948038 1331 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:16.970127 31058 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:16.951668 1568 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:16.973680 1007 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:16.987874 31165 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:16.989621 31165 raft_consensus.cc:595] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:16.990013 31165 raft_consensus.cc:603] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:16.977123 1014 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:16.972394 1741 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:16.956707 1720 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:16.970824 699 raft_consensus.cc:686] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:16.986150 1568 raft_consensus.cc:686] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:16.978305 1493 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.040158 31165 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:17.040827 31165 raft_consensus.cc:595] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:17.041182 31165 raft_consensus.cc:603] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:17.042555 699 consensus_queue.cc:237] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.043839 1621 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.045512 1621 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.046663 1439 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:17.053182 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 339094us
I20260809 07:33:16.995376 1014 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.057613 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:17.062036 1507 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260809 07:33:17.062495 1568 consensus_queue.cc:237] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.026985 1696 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:17.067981 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:17.086935 31051 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:17.087399 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:17.088723 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:17.089690 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.090296 31051 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:17.093045 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:17.096781 30911 raft_consensus.cc:1261] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:17.089340 30909 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:17.066182 1706 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:17.101524 31165 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:17.114890 31165 raft_consensus.cc:595] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:17.102435 30898 raft_consensus.cc:2379] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:17.116361 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:17.069321 1439 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:17.104804 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:17.116945 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:17.117416 31057 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:17.106910 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:17.106617 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:17.118992 31051 raft_consensus.cc:2379] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:17.111002 31054 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:17.111567 1578 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.115031 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.116976 30898 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:17.123893 30898 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:17.124953 30898 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:17.127493 31170 raft_consensus.cc:2396] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 07:33:17.129652 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:17.084756 1745 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:17.090723 1521 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:17.084465 1568 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:17.108765 1435 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.102102 1621 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:17.140720 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.148557 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.114250 1288 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.078414 1696 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.160199 1449 raft_consensus.cc:3032] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:17.160588 1568 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.101387 1281 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:17.165583 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.184141 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.185804 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.184754 31165 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:17.187923 31165 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "2c9de56c3fa3487c9dac990675c34be5"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:17.188472 31165 raft_consensus.cc:595] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Received request to transfer leadership to TS 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:17.188781 31165 raft_consensus.cc:603] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:17.178550 1521 raft_consensus.cc:979] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: : Instructing follower a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:17.190989 31006 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.171679 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.139292 1761 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:17.175969 1696 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.145299 1706 raft_consensus.cc:2741] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 07:33:17.194525 31057 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:17.147812 1745 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.162778 1214 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.161932 1720 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.180070 31051 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:17.141732 1578 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:17.192171 31170 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:17.201720 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:33:17.202464 31006 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.193203 31054 raft_consensus.cc:1226] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:17.127511 1439 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.178166 1014 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.150846 1497 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:17.197265 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:17.179510 1449 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:17.180534 1621 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:17.204901 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.209991 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:17.225239 31048 raft_consensus.cc:2379] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:17.210218 1214 consensus_queue.cc:1046] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:17.210417 1578 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.210764 1751 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260809 07:33:17.235581 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.199147 1761 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.205734 1331 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:17.238468 31054 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:17.190295 1288 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260809 07:33:17.201212 1281 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:17.239086 30909 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:17.243006 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.243968 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:17.220337 1521 raft_consensus.cc:1067] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Signalling peer a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:17.247202 1439 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.248711 1578 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.249372 1621 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.265002 1568 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.234802 1720 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:17.265974 1720 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:17.266645 1720 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:17.252158 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.252586 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.269157 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.253620 30852 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.199987 1529 consensus_queue.cc:1046] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260809 07:33:17.255237 1725 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:17.256543 1773 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
I20260809 07:33:17.256687 1717 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.260118 1288 consensus_queue.cc:1225] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260809 07:33:17.260733 1767 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
W20260809 07:33:17.261610 1730 log.cc:924] Time spent T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.324s user 0.001s sys 0.001s
I20260809 07:33:17.271415 1761 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:17.269538 31165 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:17.276013 31054 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:17.234222 1497 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.293318 1497 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:17.285856 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:17.286401 1717 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.268362 1281 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:17.294103 1281 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.295140 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:17.296658 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.279095 1439 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:17.305604 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.320278 31054 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:17.276889 1725 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.280169 1332 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.307303 1288 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:17.347015 1288 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:17.347254 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1055) took 1042 ms. Trace:
I20260809 07:33:17.347446 1288 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.347674 31182 rpcz_store.cc:276] 0809 07:33:16.304650 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.305635 (+ 985us) service_pool.cc:227] Handling call
0809 07:33:16.305998 (+ 363us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:16.618523 (+312525us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:17.346990 (+728467us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540004d3d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:17.347191 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":512832896}
I20260809 07:33:17.350471 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:17.351824 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.331110 1435 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260809 07:33:17.353487 1435 consensus_queue.cc:1225] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260809 07:33:17.358877 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:17.310751 1497 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:17.325840 1717 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:17.379594 1686 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 07:33:17.381439 1578 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:17.354139 1435 raft_consensus.cc:3032] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:17.303746 1439 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:17.385974 1435 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:17.403872 1435 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.405452 1435 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.367053 1214 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:17.406915 1214 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.412097 1332 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.420158 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.421027 31164 raft_consensus.cc:2396] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:17.422111 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.410074 1214 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.375506 1568 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:17.378594 1717 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:17.429736 30909 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260809 07:33:17.413214 1439 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.382067 1578 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.448812 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.438781 1568 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.451895 1568 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.419162 1725 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:17.457442 1766 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:17.458966 1711 log.cc:924] Time spent T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 1.053s user 0.001s sys 0.002s
I20260809 07:33:17.459542 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.383519 1497 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.440568 1717 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:17.475878 31164 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:17.482439 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:17.481482 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 294504us
I20260809 07:33:17.483884 31164 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 07:33:17.483803 31120 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.486706 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:17.510779 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:17.505765 1761 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:17.497395 1504 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:17.476629 1766 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.550693 1504 raft_consensus.cc:686] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:17.554275 1504 consensus_queue.cc:237] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.529829 1761 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:17.567762 1761 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.553771 1449 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.588876 1559 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:17.589457 1559 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:17.585790 1720 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:17.596944 1729 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.747s user 0.002s sys 0.001s
I20260809 07:33:17.602932 1492 raft_consensus.cc:3032] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:17.603430 1492 raft_consensus.cc:729] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:17.604235 1492 consensus_queue.cc:260] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:17.605538 1492 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:17.616307 1756 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.562s user 0.000s sys 0.004s
I20260809 07:33:17.512494 31164 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:18.061772 31057 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:17.902184 1717 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.242031 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:18.243302 31164 raft_consensus.cc:2396] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 07:33:18.151990 1208 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.244379 1717 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.244725 1208 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:18.152275 1439 consensus_meta.cc:287] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.697s user 0.004s sys 0.008s
I20260809 07:33:18.246156 30898 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:18.246830 1439 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.249352 1439 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.152628 1507 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.250025 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1212) took 1042 ms. Trace:
I20260809 07:33:18.153957 1497 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.250536 31051 rpcz_store.cc:276] 0809 07:33:17.207805 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.413578 (+205773us) service_pool.cc:227] Handling call
0809 07:33:17.435182 (+ 21604us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:18.058229 (+623047us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:18.058350 (+ 121us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:18.058435 (+ 85us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:18.058482 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.249537 (+191055us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b5400012cd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.249584 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.249771 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1431217408}
I20260809 07:33:17.982286 1761 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:18.053876 30891 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s user 0.003s sys 0.003s
W20260809 07:33:18.256059 30891 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.090s user 0.004s sys 0.004s
I20260809 07:33:18.256989 1761 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.250429 1507 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.260658 1507 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.111881 1214 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:18.054240 30906 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.766s user 0.005s sys 0.002s
I20260809 07:33:18.159302 1014 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:18.164597 1506 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.167594 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:18.172099 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.157828 1449 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:18.246135 1614 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
W20260809 07:33:18.264360 30906 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.977s user 0.005s sys 0.002s
I20260809 07:33:18.264891 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.265847 30899 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:17.533454 1007 consensus_meta.cc:287] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.541s user 0.013s sys 0.000s
I20260809 07:33:18.178058 1705 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:18.266906 1007 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:18.266633 1214 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.174924 1685 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:18.268561 1685 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.181533 1633 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:18.182644 1751 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.183743 1521 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.269717 1633 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.270152 1521 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.270053 1751 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.272382 1633 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.272827 1745 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.273108 1751 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.184367 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1034) took 1968 ms. Trace:
I20260809 07:33:18.269658 1506 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.272537 1521 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.274381 1685 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.269748 1014 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.251048 1497 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.276506 1497 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.277534 1014 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.190022 1496 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.174937 1549 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.016s
I20260809 07:33:18.191555 1621 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.191601 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:18.215440 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1047) took 1937 ms. Trace:
I20260809 07:33:18.219220 597 raft_consensus.cc:3032] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:18.220343 823 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.228483 1267 raft_consensus.cc:3032] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:18.230281 699 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:18.265730 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1058) took 1170 ms. Trace:
I20260809 07:33:18.267232 1705 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.294955 699 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.296100 1705 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.296792 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1204) took 1108 ms. Trace:
I20260809 07:33:18.275701 1506 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.278440 1496 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.286814 1521 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:18.297158 31054 rpcz_store.cc:276] 0809 07:33:17.188120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.321726 (+133606us) service_pool.cc:227] Handling call
0809 07:33:17.415133 (+ 93407us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:17.415329 (+ 196us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.415354 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:17.415401 (+ 47us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:17.415429 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:17.415481 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.296443 (+880962us) spinlock_profiling.cc:243] Waited 93.1 ms on lock 0x7b5400109350. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:18.296600 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":214035712}
I20260809 07:33:18.299533 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:18.299618 1496 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.274101 30904 rpcz_store.cc:276] 0809 07:33:16.216296 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.220557 (+ 4261us) service_pool.cc:227] Handling call
0809 07:33:16.229925 (+ 9368us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.553370 (+323445us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.183764 (+1630394us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540045fbd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.184153 (+ 389us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":626740480}
I20260809 07:33:18.294275 30906 rpcz_store.cc:276] 0809 07:33:17.094973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.095077 (+ 104us) service_pool.cc:227] Handling call
0809 07:33:17.095400 (+ 323us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:17.110140 (+ 14740us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.110174 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:17.111502 (+ 1328us) log.cc:842] Serialized 26 byte log entry
0809 07:33:17.282064 (+170562us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:18.264859 (+982795us) log.cc:842] Serialized 14 byte log entry
0809 07:33:18.265272 (+ 413us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.265374 (+ 102us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:18.265459 (+ 85us) raft_consensus.cc:1671] finished
0809 07:33:18.265470 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.265677 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":284941,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":24879,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":19050,"thread_start_us":1147169,"threads_started":2,"wal-append.queue_time_us":368754}
I20260809 07:33:18.304075 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1050) took 2024 ms. Trace:
I20260809 07:33:18.304456 31169 rpcz_store.cc:276] 0809 07:33:16.279226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.281725 (+ 2499us) service_pool.cc:227] Handling call
0809 07:33:18.303697 (+2021972us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54000c5154. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:18.303908 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":400882048}
I20260809 07:33:18.305155 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1181) took 1932 ms. Trace:
I20260809 07:33:18.305542 31058 rpcz_store.cc:276] 0809 07:33:16.372596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.405962 (+ 33366us) service_pool.cc:227] Handling call
0809 07:33:16.428569 (+ 22607us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.974089 (+545520us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.304703 (+1330614us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54000302d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.304967 (+ 264us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1245058944}
I20260809 07:33:18.180953 1635 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.278695 1621 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.309056 31054 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:18.293598 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1036) took 2014 ms. Trace:
I20260809 07:33:18.295365 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1071) took 2229 ms. Trace:
I20260809 07:33:18.299602 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1188) took 1925 ms. Trace:
W20260809 07:33:18.287298 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 176188us
I20260809 07:33:18.294008 1267 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:18.279606 597 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:18.306703 1635 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.358721 1438 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260809 07:33:18.364378 31046 rpcz_store.cc:276] 0809 07:33:16.065696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.067209 (+ 1513us) service_pool.cc:227] Handling call
0809 07:33:16.116561 (+ 49352us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.116757 (+ 196us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:16.371252 (+254495us) log.cc:842] Serialized 14 byte log entry
0809 07:33:18.294537 (+1923285us) spinlock_profiling.cc:243] Waited 963 us on lock 0x7b180011ecc0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7
0809 07:33:18.294699 (+ 162us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:18.294797 (+ 98us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:18.294911 (+ 114us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.295016 (+ 105us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.295183 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22212,"prepare.queue_time_us":61976,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":23351,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":12217,"spinlock_wait_cycles":2214784,"thread_start_us":253658,"threads_started":2}
I20260809 07:33:18.173944 1566 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.311208 823 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.364629 30896 rpcz_store.cc:276] 0809 07:33:16.278894 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.279621 (+ 727us) service_pool.cc:227] Handling call
0809 07:33:16.294413 (+ 14792us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.400587 (+106174us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:16.422600 (+ 22013us) log.cc:842] Serialized 14 byte log entry
0809 07:33:16.788716 (+366116us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:16.788771 (+ 55us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:16.788816 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:17.596358 (+807542us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:18.278971 (+682613us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400281c54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.279047 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.293418 (+ 14371us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":22139,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":12610,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":43225,"spinlock_wait_cycles":250145280,"thread_start_us":365689,"threads_started":2,"wal-append.queue_time_us":352010}
I20260809 07:33:18.368499 597 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.369544 823 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.369730 597 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:18.370684 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:18.297344 699 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.358222 1493 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:18.371569 31046 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:18.372816 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:18.319437 1578 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.322698 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36492 (request call id 1213) took 1114 ms. Trace:
I20260809 07:33:18.374749 31057 rpcz_store.cc:276] 0809 07:33:17.208107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.242678 (+ 34571us) service_pool.cc:227] Handling call
0809 07:33:18.322387 (+1079709us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400012554. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:18.322569 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1843873664}
I20260809 07:33:18.375422 1635 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.279350 31181 rpcz_store.cc:276] 0809 07:33:16.278379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.280222 (+ 1843us) service_pool.cc:227] Handling call
0809 07:33:16.293151 (+ 12929us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.313520 (+ 20369us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.214870 (+1901350us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5400269dd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.215213 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":45165312}
I20260809 07:33:18.376237 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:18.376890 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:18.323412 1544 raft_consensus.cc:3032] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:18.377156 31057 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:18.324448 1007 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:18.367333 1566 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.378931 1007 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.310300 1621 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.330950 1720 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.332103 1542 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:18.365298 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:18.381994 1542 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.383848 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:18.384325 1542 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.385120 1521 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:18.269294 1214 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.385604 1521 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:18.311443 1331 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.269212 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:18.385174 1621 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:18.386209 1331 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:18.387461 1621 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:18.387842 1621 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:18.388157 1621 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:18.343008 30741 catalog_manager.cc:5495] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 5, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.367424 1267 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.385623 1517 consensus_queue.cc:1046] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260809 07:33:18.307332 31058 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:18.391944 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1214) took 1183 ms. Trace:
I20260809 07:33:18.346266 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1013) took 1223 ms. Trace:
I20260809 07:33:18.347214 1766 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.392270 31058 rpcz_store.cc:276] 0809 07:33:17.208353 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:18.306512 (+1098159us) service_pool.cc:227] Handling call
0809 07:33:18.307031 (+ 519us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:18.391671 (+ 84640us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.391870 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:18.328071 1497 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.351852 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1211) took 1144 ms. Trace:
I20260809 07:33:18.353055 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1001) took 1900 ms. Trace:
I20260809 07:33:18.354269 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1060) took 1911 ms. Trace:
I20260809 07:33:18.253139 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:18.373377 823 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.366765 31059 rpcz_store.cc:276] 0809 07:33:16.373685 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.525403 (+151718us) service_pool.cc:227] Handling call
0809 07:33:16.526754 (+ 1351us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.620362 (+ 93608us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.299109 (+1678747us) spinlock_profiling.cc:243] Waited 92.1 ms on lock 0x7b54000131d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.299401 (+ 292us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":211744640}
I20260809 07:33:18.377652 1544 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:18.312987 1208 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.393123 1521 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:18.394222 31175 rpcz_store.cc:276] 0809 07:33:16.443056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.443499 (+ 443us) service_pool.cc:227] Handling call
0809 07:33:16.443791 (+ 292us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.471991 (+ 28200us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:16.472020 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:16.472056 (+ 36us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:16.472084 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.353715 (+1881631us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b54000c4254. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.353785 (+ 70us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.354035 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":64386432}
I20260809 07:33:18.396685 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:18.397955 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:18.349001 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1191) took 1917 ms. Trace:
I20260809 07:33:18.398523 823 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.334364 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:18.392457 31174 rpcz_store.cc:276] 0809 07:33:17.122473 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.122615 (+ 142us) service_pool.cc:227] Handling call
0809 07:33:17.151756 (+ 29141us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:17.294424 (+142668us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.294455 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:17.294499 (+ 44us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:17.294526 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.345759 (+1051233us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54000177d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.345842 (+ 83us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.346070 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":327595776}
I20260809 07:33:18.399828 31164 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:18.395483 1544 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.395387 31059 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:18.403216 1544 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:18.398576 31049 rpcz_store.cc:276] 0809 07:33:16.431310 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.717084 (+285774us) service_pool.cc:227] Handling call
0809 07:33:16.717484 (+ 400us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:16.895096 (+177612us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:16.895127 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:16.895535 (+ 408us) log.cc:842] Serialized 26 byte log entry
0809 07:33:17.026810 (+131275us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:17.026846 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.348336 (+1321490us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400344c54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.348398 (+ 62us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:18.348480 (+ 82us) raft_consensus.cc:1671] finished
0809 07:33:18.348493 (+ 13us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.348821 (+ 328us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":953,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":407543552,"thread_start_us":130960,"threads_started":1,"wal-append.queue_time_us":211608,"wal-append.run_cpu_time_us":2942,"wal-append.run_wall_time_us":1078037}
I20260809 07:33:18.306232 31169 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:18.392041 1517 raft_consensus.cc:3032] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:18.394025 31177 rpcz_store.cc:276] 0809 07:33:16.452120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.488385 (+ 36265us) service_pool.cc:227] Handling call
0809 07:33:16.488703 (+ 318us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.568949 (+ 80246us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:16.568998 (+ 49us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:16.569040 (+ 42us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:16.569069 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.352642 (+1783573us) spinlock_profiling.cc:243] Waited 80 ms on lock 0x7b5400184f54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.352703 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.352884 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":184021888}
I20260809 07:33:18.391618 30741 catalog_manager.cc:5495] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.414479 1497 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.415166 1497 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.414078 1517 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:18.416070 30741 catalog_manager.cc:5495] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 3, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 3 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.345708 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40582 (request call id 149) took 9520 ms. Trace:
I20260809 07:33:18.404871 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:18.418874 31181 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:18.417585 1517 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.399058 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:18.391299 1267 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:18.393788 31045 rpcz_store.cc:276] 0809 07:33:17.207491 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.378891 (+171400us) service_pool.cc:227] Handling call
0809 07:33:17.379253 (+ 362us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:17.456379 (+ 77126us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.456409 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:17.456527 (+ 118us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:17.456560 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.351398 (+894838us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b5400339fd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.351459 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.351691 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":176927488}
I20260809 07:33:18.419878 1517 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:18.402168 31174 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:18.412467 1007 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.393273 31058 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:36496
I20260809 07:33:18.417510 1497 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.419951 31175 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:18.421337 31058 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:33:18.423834 31058 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:18.381135 1566 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:18.431666 1559 consensus_meta.cc:287] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.841s user 0.008s sys 0.008s
I20260809 07:33:18.385473 31054 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:18.450162 31051 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:18.439831 30741 catalog_manager.cc:5495] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.461797 823 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.476922 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:18.448148 1559 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.489894 31046 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:18.467788 1566 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260809 07:33:18.515432 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:18.555050 1492 consensus_meta.cc:287] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.949s user 0.019s sys 0.004s
W20260809 07:33:18.559170 1288 consensus_meta.cc:287] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.127s user 0.012s sys 0.007s
W20260809 07:33:18.596483 30911 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s user 0.009s sys 0.019s
I20260809 07:33:18.418457 30745 rpcz_store.cc:276] 0809 07:33:08.824808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:08.831883 (+ 7075us) service_pool.cc:227] Handling call
0809 07:33:18.345418 (+9513535us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:13.087465 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:13.088011 (+ 546us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:13.088051 (+ 40us) write_op.cc:435] Acquired schema lock
0809 07:33:13.088068 (+ 17us) tablet.cc:660] Decoding operations
0809 07:33:13.148031 (+ 59963us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:13.160118 (+ 12087us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:13.162991 (+ 2873us) tablet.cc:683] Acquiring locks for 17 operations
0809 07:33:13.243286 (+ 80295us) tablet.cc:699] Row locks acquired
0809 07:33:13.243311 (+ 25us) write_op.cc:260] PREPARE: finished
0809 07:33:13.243455 (+ 144us) write_op.cc:270] Start()
0809 07:33:13.243568 (+ 113us) write_op.cc:276] Timestamp: P: 1786260793243410 usec, L: 0
0809 07:33:13.243589 (+ 21us) op_driver.cc:331] REPLICATION: starting
0809 07:33:13.376999 (+133410us) log.cc:842] Serialized 6698 byte log entry
0809 07:33:13.773265 (+396266us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:14.060511 (+287246us) op_driver.cc:444] REPLICATION: finished
0809 07:33:14.402468 (+341957us) write_op.cc:301] APPLY: starting
0809 07:33:14.423430 (+ 20962us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:14.491653 (+ 68223us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:14.495596 (+ 3943us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:14.565892 (+ 70296us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:14.566131 (+ 239us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0809 07:33:14.566174 (+ 43us) write_op.cc:312] APPLY: finished
0809 07:33:14.566528 (+ 354us) log.cc:842] Serialized 118 byte log entry
0809 07:33:14.566697 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:14.566854 (+ 157us) write_op.cc:454] Released schema lock
0809 07:33:14.566908 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:14.567169 (+ 261us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:14.567194 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":135553,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":282686,"apply.run_cpu_time_us":4623,"apply.run_wall_time_us":176209,"num_ops":17,"prepare.queue_time_us":389448,"prepare.run_cpu_time_us":16331,"prepare.run_wall_time_us":468711,"raft.queue_time_us":519398,"raft.run_cpu_time_us":89203,"raft.run_wall_time_us":484608,"replication_time_us":793368,"spinlock_wait_cycles":29895552,"thread_start_us":760749,"threads_started":4,"wal-append.queue_time_us":214118,"wal-append.run_cpu_time_us":17169,"wal-append.run_wall_time_us":2237095}]]}
I20260809 07:33:18.595255 30741 catalog_manager.cc:5495] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 3 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:33:18.602483 30849 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.166s user 0.026s sys 0.043s
W20260809 07:33:18.639700 30849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.219s user 0.031s sys 0.043s
W20260809 07:33:18.600375 1663 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.176s user 0.053s sys 0.073s
I20260809 07:33:18.639144 31164 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:18.649128 31175 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:18.643591 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.158s user 0.014s sys 0.041s
W20260809 07:33:18.654699 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.222s user 0.014s sys 0.041s
I20260809 07:33:18.622378 1521 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:18.655807 1663 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.273s user 0.054s sys 0.079s
W20260809 07:33:18.654282 30849 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 1.25852e+06us
I20260809 07:33:18.669914 31058 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.676712 31058 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.682395 1506 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:18.682878 1506 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:18.684067 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1054) took 1594 ms. Trace:
I20260809 07:33:18.683889 1506 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.667790 1559 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.636260 1288 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.688841 31181 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:18.678737 30741 catalog_manager.cc:5495] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 4 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:33:18.682435 1281 consensus_meta.cc:287] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 1.379s user 0.011s sys 0.006s
W20260809 07:33:18.692278 30890 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.303s user 0.013s sys 0.024s
W20260809 07:33:18.700567 30890 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.413s user 0.013s sys 0.024s
W20260809 07:33:18.700050 31048 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s user 0.007s sys 0.025s
I20260809 07:33:18.703296 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:18.706303 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 72904.7us
W20260809 07:33:18.695499 1750 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.133s user 0.039s sys 0.090s
W20260809 07:33:18.709815 31048 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.279s user 0.008s sys 0.025s
I20260809 07:33:18.684427 30891 rpcz_store.cc:276] 0809 07:33:17.090015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.091479 (+ 1464us) service_pool.cc:227] Handling call
0809 07:33:17.109615 (+ 18136us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:17.121859 (+ 12244us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.121890 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:17.122542 (+ 652us) log.cc:842] Serialized 52 byte log entry
0809 07:33:18.256779 (+1134237us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:18.257405 (+ 626us) log.cc:842] Serialized 14 byte log entry
0809 07:33:18.257844 (+ 439us) log.cc:842] Serialized 14 byte log entry
0809 07:33:18.257968 (+ 124us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.258119 (+ 151us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:18.309494 (+ 51375us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b48000d26d8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:18.683860 (+374366us) raft_consensus.cc:1671] finished
0809 07:33:18.683890 (+ 30us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.684013 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11849,"prepare.queue_time_us":685751,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":20679,"raft.run_cpu_time_us":2439,"raft.run_wall_time_us":1180560,"spinlock_wait_cycles":22921088,"thread_start_us":1090062,"threads_started":1,"wal-append.queue_time_us":1470448}
I20260809 07:33:18.704631 1506 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:18.718950 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.565s user 0.018s sys 0.048s
W20260809 07:33:18.726788 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.668s user 0.018s sys 0.048s
I20260809 07:33:18.710399 30741 catalog_manager.cc:5495] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:33:18.715291 31004 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.000s sys 0.033s
I20260809 07:33:18.705344 1705 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:18.728559 31004 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.670s user 0.001s sys 0.034s
W20260809 07:33:18.720369 1750 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.268s user 0.040s sys 0.091s
I20260809 07:33:18.737502 1195 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.738111 823 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:18.704782 1281 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.746399 1281 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.747152 1521 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:18.737490 1392 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.149s user 0.000s sys 0.042s
W20260809 07:33:18.748137 1392 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.190s user 0.001s sys 0.042s
W20260809 07:33:18.748955 1391 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.133s user 0.015s sys 0.032s
W20260809 07:33:18.749439 1391 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.178s user 0.015s sys 0.033s
I20260809 07:33:18.738327 1195 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.741456 823 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.741700 30741 catalog_manager.cc:5495] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 5 to 6, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.727823 1007 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
W20260809 07:33:18.772603 1504 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.134s user 0.001s sys 0.018s
I20260809 07:33:18.778342 31165 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:18.779567 823 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.779870 1195 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.780639 1331 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.737563 1288 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:18.785897 1504 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.229s user 0.002s sys 0.018s
I20260809 07:33:18.740651 1766 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:18.750928 1566 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:18.781234 1391 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:18.799274 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 48183us
I20260809 07:33:18.753283 1705 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.783438 1685 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:18.791260 30853 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:18.811443 31006 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:18.798197 1195 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:18.809158 1545 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.159s user 0.009s sys 0.030s
I20260809 07:33:18.808692 1578 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:18.833717 31003 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.630s user 0.009s sys 0.013s
W20260809 07:33:18.834136 31003 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.672s user 0.009s sys 0.013s
W20260809 07:33:18.845698 31172 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s user 0.006s sys 0.034s
W20260809 07:33:18.846112 31172 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s user 0.006s sys 0.034s
I20260809 07:33:18.847074 1331 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.825121 1685 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.849809 1685 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.849900 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:45792 (request call id 61) took 1343 ms. Trace:
I20260809 07:33:18.850272 31182 rpcz_store.cc:276] 0809 07:33:17.506244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.512138 (+ 5894us) service_pool.cc:227] Handling call
0809 07:33:18.848612 (+1336474us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b540004db54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7eaf0 00007f4094f01f2a 00007f4099e3ec6e 00007f4093535807 00007f4093535792 00007f40935356c1 00007f409353563c 00007f40935348f2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:18.849666 (+ 1054us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1560529664}
W20260809 07:33:18.851537 1803 log.cc:924] Time spent T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.070s user 0.000s sys 0.001s
I20260809 07:33:18.855229 1746 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:18.857419 1578 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.859113 30891 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:18.859683 1578 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.860323 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.860942 30891 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:18.861440 1724 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:18.861887 1195 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.863055 1544 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.863677 1544 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.864212 1195 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.805419 1766 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.865245 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:18.866329 1766 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.866770 1504 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:18.867142 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:18.868085 1521 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:18.867172 1504 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:18.870025 1504 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:18.870625 31055 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s user 0.003s sys 0.019s
W20260809 07:33:18.870947 31055 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s user 0.003s sys 0.019s
I20260809 07:33:18.871029 1566 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.871220 30741 catalog_manager.cc:5495] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.873262 1566 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.866411 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1070) took 1515 ms. Trace:
I20260809 07:33:18.874572 31165 rpcz_store.cc:276] 0809 07:33:17.350570 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.351564 (+ 994us) service_pool.cc:227] Handling call
0809 07:33:18.866117 (+1514553us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002a2bd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:18.866298 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3215431040}
I20260809 07:33:18.875306 1705 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.875538 1521 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:18.875672 1544 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:18.877271 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.877907 30896 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
W20260809 07:33:18.878604 1741 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.800s user 0.014s sys 0.014s
I20260809 07:33:18.878767 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:18.879057 1741 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.855s user 0.015s sys 0.015s
I20260809 07:33:18.879338 1741 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:18.880012 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.880545 1741 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.881175 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.873441 30741 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: leader changed from <none> to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.831485 1391 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.885568 1391 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:18.842660 1559 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.890748 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.885516 30741 catalog_manager.cc:5495] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.891736 1786 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
W20260809 07:33:18.846228 1811 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.057s user 0.001s sys 0.000s
W20260809 07:33:18.921722 31057 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s user 0.017s sys 0.025s
W20260809 07:33:18.949200 31057 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.569s user 0.017s sys 0.028s
I20260809 07:33:18.912087 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1205) took 1720 ms. Trace:
I20260809 07:33:18.949331 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:18.889947 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 45609us
W20260809 07:33:18.845346 1545 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.290s user 0.009s sys 0.031s
I20260809 07:33:18.861887 1724 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.967448 30741 catalog_manager.cc:5495] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 1 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:18.913542 1614 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 07:33:18.979453 31118 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.743s user 0.065s sys 0.067s
I20260809 07:33:18.925696 1746 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:18.950485 31048 rpcz_store.cc:276] 0809 07:33:17.191490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.336018 (+144528us) service_pool.cc:227] Handling call
0809 07:33:17.388130 (+ 52112us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:17.395911 (+ 7781us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.395938 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:17.430967 (+ 35029us) log.cc:842] Serialized 556 byte log entry
0809 07:33:18.739763 (+1308796us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:18.753673 (+ 13910us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.782727 (+ 29054us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:18.888220 (+105493us) raft_consensus.cc:1671] finished
0809 07:33:18.888240 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:18.912022 (+ 23782us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":1278947,"threads_started":1,"wal-append.queue_time_us":1417009}
W20260809 07:33:18.985827 31118 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.917s user 0.065s sys 0.067s
I20260809 07:33:18.983691 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.988265 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:18.986902 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:18.988433 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.993366 30891 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:18.995680 30896 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:18.996580 31006 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:18.997298 30827 auto_rebalancer.cc:663] Could not move replica: Not found: Couldn't find leader for tablet 335fccb907184174b4a6428ce262e7dc
W20260809 07:33:18.997812 30827 auto_rebalancer.cc:264] scheduled replica move failed to complete: Not found: Couldn't find leader for tablet 335fccb907184174b4a6428ce262e7dc
W20260809 07:33:19.001616 30909 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s user 0.034s sys 0.073s
W20260809 07:33:19.001971 30909 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s user 0.035s sys 0.074s
I20260809 07:33:19.003695 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.003902 1521 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:18.975286 1724 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.004416 30899 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.004525 1521 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.006676 1521 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.007390 1529 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.008035 1529 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:19.010350 30900 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s user 0.008s sys 0.020s
W20260809 07:33:19.010676 30900 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s user 0.008s sys 0.020s
I20260809 07:33:19.012243 1746 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:18.972319 1542 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.023608 30741 catalog_manager.cc:5495] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 5, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:19.018558 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:18.975852 1720 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:19.029109 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 248082us
I20260809 07:33:18.956416 1559 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.026825 30898 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.027757 30906 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.010135 1529 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.034020 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1070) took 2968 ms. Trace:
W20260809 07:33:18.979794 31005 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.681s (SENT) [suppressed 3 similar messages]
I20260809 07:33:19.038892 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:18.983143 1705 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.032248 1817 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:19.042480 32141 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.906s user 0.025s sys 0.061s
W20260809 07:33:19.042414 31170 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s user 0.013s sys 0.044s
I20260809 07:33:19.014354 1746 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:19.048558 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.708s user 0.052s sys 0.061s
W20260809 07:33:19.059046 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.874s user 0.052s sys 0.061s
W20260809 07:33:19.060976 31170 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s user 0.013s sys 0.045s
I20260809 07:33:19.062387 30904 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
W20260809 07:33:19.052716 32141 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.941s user 0.025s sys 0.061s
I20260809 07:33:19.042836 1331 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:19.068192 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:19.064723 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:19.052356 1799 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:19.048009 1281 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:19.067344 1811 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.081s user 0.001s sys 0.000s
W20260809 07:33:19.066448 1621 consensus_meta.cc:287] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.674s user 0.014s sys 0.000s
I20260809 07:33:19.104030 1621 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.036427 31055 rpcz_store.cc:276] 0809 07:33:16.065419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.067131 (+ 1712us) service_pool.cc:227] Handling call
0809 07:33:16.115995 (+ 48864us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:16.116196 (+ 201us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:18.195178 (+2078982us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b18003fd300. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fce155 00007f40988ce167 00007f4094f03807 00007f4094f12f0f
0809 07:33:18.195458 (+ 280us) log.cc:842] Serialized 14 byte log entry
0809 07:33:18.971824 (+776366us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:18.979938 (+ 8114us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:19.019143 (+ 39205us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:19.026022 (+ 6879us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.033952 (+ 7930us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":80890,"prepare.queue_time_us":52955,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":59935,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":17206,"spinlock_wait_cycles":4237184,"thread_start_us":727420,"threads_started":2}
I20260809 07:33:19.092731 31182 raft_consensus.cc:3032] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:19.095639 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:19.040764 1544 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:19.100430 31181 raft_consensus.cc:1226] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:19.103523 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1061) took 2006 ms. Trace:
I20260809 07:33:19.104171 1786 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:19.086424 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.104247 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.068064 1705 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.106138 31182 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:19.110401 30904 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:19.113823 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.089699 1529 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.115023 30853 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.121634 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.122792 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.108897 30911 rpcz_store.cc:276] 0809 07:33:17.097101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.097579 (+ 478us) service_pool.cc:227] Handling call
0809 07:33:17.097877 (+ 298us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:17.113640 (+ 15763us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:17.463891 (+350251us) log.cc:842] Serialized 14 byte log entry
0809 07:33:17.556633 (+ 92742us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b48000d26d8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:18.653187 (+1096554us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:18.714821 (+ 61634us) log.cc:842] Serialized 26 byte log entry
0809 07:33:18.724146 (+ 9325us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:18.736109 (+ 11963us) log.cc:842] Serialized 14 byte log entry
0809 07:33:18.737814 (+ 1705us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.737914 (+ 100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:18.744570 (+ 6656us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b18003d1ce0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:19.099426 (+354856us) raft_consensus.cc:1671] finished
0809 07:33:19.099448 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.103456 (+ 4008us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25736,"prepare.queue_time_us":436556,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":2746,"raft.queue_time_us":31962,"raft.run_cpu_time_us":2338,"raft.run_wall_time_us":439057,"spinlock_wait_cycles":9607424,"thread_start_us":1446445,"threads_started":2,"wal-append.queue_time_us":1247624}
I20260809 07:33:19.117292 1521 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:19.108641 1544 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:19.045390 1720 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:19.127290 31164 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:19.129151 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.129860 1799 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.129783 1529 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.135634 1544 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.136710 1720 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:19.136981 1544 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:19.125151 1517 consensus_meta.cc:287] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.705s user 0.011s sys 0.003s
I20260809 07:33:19.105648 1663 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:19.117015 1310 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:19.148582 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.150549 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.108577 1542 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.132617 31182 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.134512 1822 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:19.136632 1529 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.126199 1521 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:19.158418 1521 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.115358 1281 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.151224 31165 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:19.155532 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.156335 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.157902 1822 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:19.095532 1817 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.139432 1833 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:19.154464 1497 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:19.152565 1559 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.175062 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.191915 31182 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.209506 1830 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:19.179414 1497 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:19.178277 1332 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:19.174242 1822 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.182590 1517 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:19.169365 1663 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.210481 1621 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.167068 1504 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.211082 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.176857 1529 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:19.218684 1492 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:19.215180 1288 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:19.242501 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 67529.4us
I20260809 07:33:19.178105 1310 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.242864 1566 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:19.246083 1497 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.208475 1724 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:19.272727 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 203205us
I20260809 07:33:19.283511 31165 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:19.251129 1517 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:19.284363 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1059) took 2179 ms. Trace:
I20260809 07:33:19.196878 1833 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:19.288045 1566 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:19.286463 1807 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.177s user 0.001s sys 0.001s
I20260809 07:33:19.261427 1685 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:19.250267 1347 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:19.274410 1529 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.294823 1817 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.294744 1566 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:19.290057 1517 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:19.250242 1332 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.302392 1288 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.295003 1821 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.288666 30890 rpcz_store.cc:276] 0809 07:33:17.095225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.095595 (+ 370us) service_pool.cc:227] Handling call
0809 07:33:17.107656 (+ 12061us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:17.116488 (+ 8832us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.116517 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:17.117045 (+ 528us) log.cc:842] Serialized 52 byte log entry
0809 07:33:17.287064 (+170019us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:17.600986 (+313922us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b48000d26d8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:18.713550 (+1112564us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.234013 (+520463us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.246657 (+ 12644us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:19.261169 (+ 14512us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:19.274224 (+ 13055us) raft_consensus.cc:1671] finished
0809 07:33:19.274261 (+ 37us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.274424 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":59906,"prepare.queue_time_us":303396,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":55900,"raft.run_cpu_time_us":1840,"raft.run_wall_time_us":477766,"spinlock_wait_cycles":77637888,"thread_start_us":1962144,"threads_started":3,"wal-append.queue_time_us":278952,"wal-append.run_cpu_time_us":2651,"wal-append.run_wall_time_us":1004656}
I20260809 07:33:19.306072 1529 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:19.311412 1824 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.071s user 0.001s sys 0.000s
W20260809 07:33:19.308977 1826 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
I20260809 07:33:19.342021 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.296798 1492 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.307912 1705 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.350531 30904 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.283880 1521 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.357183 1621 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.332101 1542 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.363859 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.344204 1799 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.365693 30911 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.348716 1281 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.311980 1685 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.373626 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.384969 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.389524 30911 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:19.366036 1786 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:19.387681 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 407907us
I20260809 07:33:19.334398 1347 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.380883 1578 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:19.398393 1817 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:19.398876 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:19.380692 1331 raft_consensus.cc:686] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:19.403719 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.399444 30890 raft_consensus.cc:1261] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:19.352207 1821 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:19.419530 597 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:19.382853 1621 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.420063 597 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:19.420066 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.421468 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.428345 31046 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:19.429637 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.420171 1724 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.417415 1504 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.430860 1820 consensus_queue.cc:1046] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260809 07:33:19.433036 1685 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.435742 1332 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.436118 1833 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.401572 1720 raft_consensus.cc:465] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Not starting pre-election -- already a leader
W20260809 07:33:19.411118 1829 log.cc:924] Time spent T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.201s user 0.002s sys 0.000s
W20260809 07:33:19.430104 1506 consensus_meta.cc:287] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.702s user 0.010s sys 0.000s
I20260809 07:33:19.438184 1331 consensus_queue.cc:237] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.448575 1007 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:19.449040 597 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.449182 1007 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:19.451429 1828 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.198s user 0.001s sys 0.000s
I20260809 07:33:19.451771 1007 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.443732 1817 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.453583 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:19.441264 1578 raft_consensus.cc:686] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:19.455309 1817 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.455199 1578 consensus_queue.cc:237] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.456416 1820 raft_consensus.cc:3032] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:19.456807 1820 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:19.453167 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1022) took 2038 ms. Trace:
I20260809 07:33:19.454336 31054 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:19.423415 1559 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.460111 31164 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:19.410707 1281 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260809 07:33:19.465529 31058 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:19.459079 31170 rpcz_store.cc:276] 0809 07:33:17.414772 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.423389 (+ 8617us) service_pool.cc:227] Handling call
0809 07:33:17.423737 (+ 348us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:17.424141 (+ 404us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.424170 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:17.424907 (+ 737us) log.cc:842] Serialized 52 byte log entry
0809 07:33:19.103283 (+1678376us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:19.103345 (+ 62us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:19.103442 (+ 97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:19.441568 (+338126us) raft_consensus.cc:1671] finished
0809 07:33:19.441586 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.453072 (+ 11486us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4540,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4538,"thread_start_us":892815,"threads_started":1,"wal-append.queue_time_us":1654127}
I20260809 07:33:19.416000 1267 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260809 07:33:19.408913 1786 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.471947 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.472553 31058 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.445421 1809 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:19.462836 1837 log.cc:924] Time spent T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.086s user 0.001s sys 0.000s
I20260809 07:33:19.477685 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.477725 1288 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.478657 1347 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.479287 1827 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:19.512909 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 102162us
I20260809 07:33:19.481678 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:19.482152 1800 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:19.483081 31051 raft_consensus.cc:1261] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260809 07:33:19.513893 31045 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.483101 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1063) took 3024 ms. Trace:
I20260809 07:33:19.484746 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1063) took 2387 ms. Trace:
I20260809 07:33:19.485697 1621 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.472569 1786 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.524088 1809 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.526737 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.528234 31051 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:19.486577 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.488145 31058 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:19.488579 1506 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:19.533870 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.489780 1559 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.534785 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.489008 1529 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.535955 1559 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.491230 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.536723 1529 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.506062 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:19.481415 1825 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.261s user 0.001s sys 0.002s
I20260809 07:33:19.537461 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.538282 31046 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:19.538225 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.538594 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1071) took 1368 ms. Trace:
I20260809 07:33:19.540589 1310 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.540820 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.541221 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.539136 31049 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.466722 1391 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:19.542450 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:19.543584 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:19.543998 1845 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.085s user 0.001s sys 0.000s
I20260809 07:33:19.506505 1332 consensus_queue.cc:1046] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260809 07:33:19.543591 31054 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:19.503140 1521 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.496364 1821 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.514623 30900 rpcz_store.cc:276] 0809 07:33:17.097622 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:17.119465 (+ 21843us) service_pool.cc:227] Handling call
0809 07:33:17.126766 (+ 7301us) raft_consensus.cc:1457] Updating replica for 3 ops
0809 07:33:17.127647 (+ 881us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:17.127673 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 3 ops
0809 07:33:17.128321 (+ 648us) log.cc:842] Serialized 78 byte log entry
0809 07:33:18.177953 (+1049632us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b18003d1d40. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411 00007f4094ecd5cf
0809 07:33:18.221833 (+ 43880us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b48000d26d8. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:19.011151 (+789318us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:19.011520 (+ 369us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.053007 (+ 41487us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.086629 (+ 33622us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.181395 (+ 94766us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b18003d1d40. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c990 00007f4091fce864 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7 00007f4094f14058
0809 07:33:19.189327 (+ 7932us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:19.218627 (+ 29300us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:19.473058 (+254431us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b18003d1d40. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1764 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:19.484540 (+ 11482us) raft_consensus.cc:1671] finished
0809 07:33:19.484566 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.484692 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":140678,"prepare.queue_time_us":15246,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":132,"raft.queue_time_us":174016,"raft.run_cpu_time_us":2685,"raft.run_wall_time_us":1127156,"spinlock_wait_cycles":39123712,"thread_start_us":832823,"threads_started":1,"wal-append.queue_time_us":2056863}
I20260809 07:33:19.545070 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.519193 30853 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.545909 31054 raft_consensus.cc:2379] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 4: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:19.546464 1332 raft_consensus.cc:3032] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:19.541978 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.521822 1827 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.514516 31172 rpcz_store.cc:276] 0809 07:33:16.458915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:16.558878 (+ 99963us) service_pool.cc:227] Handling call
0809 07:33:16.560233 (+ 1355us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:18.211666 (+1651433us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:18.211714 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:18.212946 (+ 1232us) log.cc:842] Serialized 556 byte log entry
0809 07:33:18.846542 (+633596us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:18.846602 (+ 60us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:18.850748 (+ 4146us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b540004db54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:18.850799 (+ 51us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:19.482726 (+631927us) raft_consensus.cc:1671] finished
0809 07:33:19.482756 (+ 30us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.483022 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3549983488,"thread_start_us":633176,"threads_started":1,"wal-append.queue_time_us":925562}
I20260809 07:33:19.547770 1391 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:19.547333 30853 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.530965 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.549110 1827 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.532541 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:19.467839 1822 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.534041 1506 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:19.538960 30909 rpcz_store.cc:276] 0809 07:33:18.170393 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:18.171844 (+ 1451us) service_pool.cc:227] Handling call
0809 07:33:18.172602 (+ 758us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:18.173379 (+ 777us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:18.173409 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:18.174201 (+ 792us) log.cc:842] Serialized 26 byte log entry
0809 07:33:19.002294 (+828093us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:19.002360 (+ 66us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:19.002422 (+ 62us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:19.480868 (+478446us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:19.538286 (+ 57418us) raft_consensus.cc:1671] finished
0809 07:33:19.538306 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.538538 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":12453,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":4033,"spinlock_wait_cycles":26602368,"thread_start_us":827039,"threads_started":1,"wal-append.queue_time_us":991076}
I20260809 07:33:19.542589 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:19.556115 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 107996us
I20260809 07:33:19.556301 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.557044 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.497645 31182 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:19.557416 1827 consensus_queue.cc:1046] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:19.557700 31006 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:19.557425 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.526707 1517 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.541244 1492 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.457844 1820 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.559144 31049 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.547341 31055 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.549840 31169 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.546948 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.533536 31045 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:19.550727 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1216) took 2157 ms. Trace:
I20260809 07:33:19.539223 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.543051 31058 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.549103 31181 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:19.552538 30906 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.530438 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.558516 1827 raft_consensus.cc:3032] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:19.550259 31059 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.564952 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:19.537324 1836 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.150s user 0.001s sys 0.001s
I20260809 07:33:19.506811 1545 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:19.570737 1545 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:19.567428 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.565367 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.560473 30853 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.559926 31006 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:19.566965 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.569650 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:19.489794 1566 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.569893 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.507890 1663 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.546922 1332 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:19.570364 1854 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:19.526274 1809 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.571733 1545 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.595587 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.623620 1497 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.558818 1746 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:19.565474 1493 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:19.567756 1521 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.073s
I20260809 07:33:19.561120 31057 rpcz_store.cc:276] 0809 07:33:17.393689 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:18.378893 (+985204us) service_pool.cc:227] Handling call
0809 07:33:18.379339 (+ 446us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:18.379921 (+ 582us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 07:33:18.950034 (+570113us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.397149 (+447115us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:19.491410 (+ 94261us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b2c00000238. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:33:19.491994 (+ 584us) log.cc:842] Serialized 52 byte log entry
0809 07:33:19.493204 (+ 1210us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:19.496471 (+ 3267us) log.cc:842] Serialized 14 byte log entry
0809 07:33:19.496666 (+ 195us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:19.496752 (+ 86us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:19.512233 (+ 15481us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b180035d580. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:19.550526 (+ 38293us) raft_consensus.cc:1671] finished
0809 07:33:19.550546 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:19.550674 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":77968,"prepare.queue_time_us":811960,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":7336,"raft.queue_time_us":59728,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":330,"spinlock_wait_cycles":20961024,"thread_start_us":928175,"threads_started":2,"wal-append.queue_time_us":437259}
I20260809 07:33:19.564951 1827 raft_consensus.cc:729] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:19.571786 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.606237 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.630232 31058 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.637470 1854 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:19.647373 1854 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:19.650635 597 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:19.651206 597 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.652841 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.653084 597 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:19.653609 31055 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:19.654402 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.655049 31055 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:19.655860 1517 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:19.638056 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 78196.2us
I20260809 07:33:19.658008 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:19.655825 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.639832 31049 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:19.641762 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.646742 31169 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:19.661042 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.639011 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.643200 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.638561 1347 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:19.662755 31045 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:19.665019 31045 raft_consensus.cc:1226] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:19.660158 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:19.643931 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 72220.6us
I20260809 07:33:19.665984 31046 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:19.638746 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 78320.2us
I20260809 07:33:19.637385 31054 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:19.639267 31172 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:33:19.643316 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 176183us
I20260809 07:33:19.662838 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.670634 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:19.683359 31058 raft_consensus.cc:2379] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:19.675997 31054 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:19.686507 31006 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:19.687894 31045 raft_consensus.cc:1261] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:19.675182 1854 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.690186 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:19.644040 1521 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:19.690869 1521 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:19.638921 1545 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:19.691346 31057 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:19.638150 1331 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:19.636484 1820 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.640892 1746 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.642633 1566 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:19.693938 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:19.760243 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 66383.7us
I20260809 07:33:19.761258 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.761953 30911 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:19.762259 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.762602 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.763278 30909 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.764420 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.765004 30891 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.767550 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.665620 1347 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:19.672875 1724 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:19.641220 1497 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:19.792454 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 55623.6us
I20260809 07:33:19.794605 30898 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.799598 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.800148 30906 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:19.692684 31055 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:19.803030 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.637080 1332 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.794399 1746 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.698422 1521 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.737270 1809 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:19.766233 1817 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:19.804188 31006 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:19.803303 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:19.641994 1827 consensus_queue.cc:260] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.804733 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.645794 1506 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.799149 1497 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:19.805140 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 119729us
I20260809 07:33:19.698855 1331 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:19.806676 30900 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.796160 1347 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:19.736171 1529 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:19.810753 1332 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:19.794834 1724 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:19.820007 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:19.824609 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.823786 1310 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:19.839519 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:19.842185 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:20.149849 31170 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:19.865707 30904 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:19.851136 1331 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:19.924036 1195 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:20.153798 32198 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Committing config change with OpId 6.5: config changed from index 2 to 5, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:20.154760 1195 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.155516 1724 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:20.156886 1195 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:19.843120 1857 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.144s user 0.000s sys 0.000s
I20260809 07:33:19.936685 30898 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:20.072371 1663 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260809 07:33:19.842556 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:20.107870 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:20.063982 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.995373 1544 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:19.812711 1856 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Committing config change with OpId 6.5: config changed from index 2 to 5, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:19.823163 1817 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.165216 1817 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:20.165670 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 272518us
I20260809 07:33:20.166579 1529 raft_consensus.cc:686] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:20.166777 1855 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:19.955765 30911 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:20.167160 1545 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:19.979929 30906 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:19.846665 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.171931 1506 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.174777 1529 consensus_queue.cc:237] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:20.163283 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 99407.1us
I20260809 07:33:19.892213 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:20.178980 31175 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:20.179194 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:19.813874 1281 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.180284 1506 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:20.201380 1822 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.202976 30911 raft_consensus.cc:1226] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 07:33:20.202917 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:20.205240 31175 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:20.208030 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.211149 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.211803 31048 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:20.215337 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.215598 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.217113 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.180032 1281 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:19.892931 1497 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.163080 30909 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 5.
I20260809 07:33:20.164304 1544 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.181138 1545 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:20.182976 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.183314 1854 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.112351 30900 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:20.222457 31172 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:20.182229 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.195137 30906 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260809 07:33:19.892052 1860 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.088s user 0.000s sys 0.003s
I20260809 07:33:20.200186 31181 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:19.840883 1809 raft_consensus.cc:686] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:20.220466 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.220839 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.228286 31046 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:20.221035 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.221474 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.220306 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.228518 31054 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:20.229836 30852 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.230353 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.216441 1310 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:20.233839 597 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:20.230656 1545 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.239990 1545 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.240394 1545 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.241603 1854 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:20.243660 30899 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 07:33:20.244902 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.245574 30900 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:20.226773 1855 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260809 07:33:20.247073 1521 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.249150 1521 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.250239 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.251819 31051 raft_consensus.cc:2396] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:20.252653 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.253481 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.253708 1746 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.254446 30900 raft_consensus.cc:2396] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 07:33:20.234727 1195 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.256800 1347 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.259617 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.260267 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.265919 31046 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:20.269570 1506 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:20.275252 31120 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.267481 1195 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:20.267786 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.278093 30900 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:20.275584 1506 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.279131 1195 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.279386 1506 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:20.275445 1822 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:20.279781 1310 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.271925 1521 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.285599 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.279341 1854 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:20.296886 31048 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:20.299606 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.300283 30906 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:20.300392 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.288775 1521 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.302804 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.301120 1822 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:20.305006 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.306569 1854 raft_consensus.cc:686] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:20.301016 31004 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff
I20260809 07:33:20.310855 31006 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.313251 31006 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.314073 31006 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.315846 1559 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.316255 1559 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.316591 1559 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.317032 1854 consensus_queue.cc:237] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.314675 1885 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.318686 1885 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.319097 1885 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.309300 1521 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.318627 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.320439 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.307850 1517 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:20.310703 1566 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:20.323067 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.324863 1007 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:20.325762 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.326033 1786 consensus_queue.cc:1035] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:20.321166 1838 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.325057 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.325803 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.316530 1497 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.339233 1746 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.333717 1281 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.344924 30900 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:20.323736 1517 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.325393 31170 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "17576a737982459aa79301ce4f7eb096"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:20.346669 1331 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.314719 1809 consensus_queue.cc:237] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.316604 1820 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260809 07:33:20.373831 30906 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:20.333863 1705 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:20.376951 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.329728 1838 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.356844 1559 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.361292 31170 raft_consensus.cc:595] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
W20260809 07:33:20.378963 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 63682.7us
I20260809 07:33:20.379395 31170 raft_consensus.cc:603] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260809 07:33:20.366529 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 547127us
I20260809 07:33:20.372637 1195 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.335973 1544 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.329747 1566 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.379527 30906 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:20.382557 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53820.6us
I20260809 07:33:20.366667 1517 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:20.386669 30899 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:20.389705 1895 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:20.390755 1310 raft_consensus.cc:3032] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:20.391052 1310 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:20.391553 1559 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.387292 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.394852 1544 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.395299 1544 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.395418 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.398285 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.398666 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.399117 1621 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:20.400327 1621 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:20.400740 1621 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:20.400897 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.401372 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:20.401574 30898 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:20.380112 1838 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.402567 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.403685 31175 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:20.403807 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.406831 1827 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:20.407581 1195 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.377488 1705 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:20.408202 1705 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:20.391759 1310 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.413224 1885 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.387318 1347 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.415105 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.413050 1310 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:20.419036 31175 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "51e40c75108c4a86a26730860edee159"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:20.419669 31175 raft_consensus.cc:595] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.420003 31175 raft_consensus.cc:603] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:20.416572 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.417517 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.415791 30906 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:20.395727 1544 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.422814 30899 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:20.387717 1566 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:20.424247 31004 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 123319us
I20260809 07:33:20.429479 31170 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:20.433250 30852 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.434109 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:20.437810 31175 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.441085 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.442209 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.443974 1898 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:20.444490 1898 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:20.445506 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.446276 30896 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:20.446779 30899 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:20.447962 31170 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "656029f31f2846908145dfacac4abdf2"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:20.448550 31170 raft_consensus.cc:595] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.448892 31170 raft_consensus.cc:603] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:20.447741 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.449818 30890 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.450860 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.440919 1887 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:20.451472 32141 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:20.451882 1887 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.451534 30909 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:20.447713 1898 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.450371 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.454185 30896 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:20.434675 1817 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.451841 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.441053 1347 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:20.456288 1817 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.457031 1347 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:20.457904 1347 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.432889 1506 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.458506 1724 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.459376 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.460143 1506 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.460440 1724 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.460983 30896 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:20.461305 30896 pending_rounds.cc:76] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Aborting all ops after (but not including) 2
I20260809 07:33:20.461536 30896 pending_rounds.cc:97] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 07:33:20.461808 30896 raft_consensus.cc:2866] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:20.464109 1545 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.468987 30890 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:20.469538 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.471030 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.471649 31054 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:20.472687 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.473091 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.473793 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.473666 31046 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:20.473260 31045 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.479245 31175 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:20.468209 1809 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260809 07:33:20.481756 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.482520 1559 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:20.482924 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.483222 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.488919 31182 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:20.489794 31046 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:20.437343 1746 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260809 07:33:20.490195 1621 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.490584 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:20.533252 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52249.8us
I20260809 07:33:20.498010 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.552459 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:20.577073 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 91066.3us
I20260809 07:33:20.492991 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.803828 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.812381 31054 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.528283 31045 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:20.813789 31182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "2c9de56c3fa3487c9dac990675c34be5"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:20.494552 1545 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.482878 1559 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:20.814491 31182 raft_consensus.cc:595] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Received request to transfer leadership to TS 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:20.792488 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 122076us
I20260809 07:33:20.814985 31182 raft_consensus.cc:603] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:20.803483 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.818223 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.821292 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.822172 30898 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:20.457931 1887 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.815675 30852 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.825312 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.828891 31045 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:20.830169 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.830745 31046 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:20.831317 31006 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.691843 30911 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:20.732352 1854 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:20.836251 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:20.840772 31175 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:20.491416 1822 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:20.841468 1559 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.843212 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:20.839285 31059 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:20.545279 1897 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:20.458541 1521 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.842590 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:20.847512 31169 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.603964 1331 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.841583 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:20.850855 31172 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:20.852077 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:20.853183 31048 raft_consensus.cc:2396] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:20.628959 1529 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:20.533463 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 82093.1us
I20260809 07:33:20.849139 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:20.849611 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:20.858821 31182 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:20.859037 30904 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:20.646540 1885 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:20.853967 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.839038 1663 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:20.493435 1838 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.640640 1566 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.837934 1332 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:20.636761 1517 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.692972 1705 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.862185 31046 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:20.869402 1663 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.871996 31054 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:20.872797 31049 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Committing config change with OpId 6.5: config changed from index 2 to 5, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:20.874526 1817 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.848781 1902 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:20.878026 31170 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:20.639941 1544 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.849117 1898 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:20.883940 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:20.888845 31181 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:20.893374 31169 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:20.896978 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:20.898350 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:20.919060 31051 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:20.848166 1854 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.865157 1529 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:20.929670 31164 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:21.045399 30899 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:21.177371 30904 raft_consensus.cc:2379] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:20.996903 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 178731us
I20260809 07:33:20.883419 1332 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:21.179147 30898 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 7
I20260809 07:33:21.179481 1332 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:21.006718 1857 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.108s user 0.000s sys 0.000s
I20260809 07:33:21.180982 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.950836 31182 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:21.183017 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 299148us
I20260809 07:33:21.135893 1822 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.184338 1544 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.184150 1822 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.184692 30852 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.185612 1195 raft_consensus.cc:2741] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 07:33:21.186368 1195 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:21.186635 1822 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.186729 1195 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:21.187178 31054 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:21.006920 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.151593 1517 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:20.982263 1902 raft_consensus.cc:686] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:21.124106 1911 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.079s user 0.000s sys 0.001s
I20260809 07:33:20.954981 31057 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:21.200732 31172 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:21.150975 1705 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.188514 31181 raft_consensus.cc:595] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.202705 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.203287 30906 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.190423 1898 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:21.151880 1521 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.186745 31165 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:21.191257 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.188199 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.191767 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.182494 1926 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:21.195055 30900 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:21.170403 1922 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.101s user 0.001s sys 0.000s
I20260809 07:33:21.208874 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:20.900151 1817 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:21.195369 1007 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.204668 30898 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:21.208508 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.207842 30890 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:21.210136 31006 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.210736 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.211099 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.192196 1663 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.212097 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.196537 1195 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.215785 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.216852 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.207682 1898 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.218047 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:33:21.204748 1856 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:21.223672 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.223769 1800 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:21.224097 1917 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
I20260809 07:33:21.194137 1927 consensus_queue.cc:1046] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260809 07:33:21.229012 31055 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:21.259984 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:21.260617 31055 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:21.209196 1926 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:21.229663 1902 consensus_queue.cc:237] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.217571 1521 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:21.204053 1529 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.213645 1705 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:21.216842 1817 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.217073 1822 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:21.264549 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:21.267647 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 50256.1us
I20260809 07:33:21.222041 1517 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:21.222568 1895 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:21.274156 30906 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:21.219607 1566 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.214910 1347 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260809 07:33:21.278379 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.263281 1901 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:21.266242 1926 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.278852 1521 raft_consensus.cc:686] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:21.284626 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.217846 1007 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.287804 30904 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:21.272763 1663 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 07:33:21.294488 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 84428.5us
I20260809 07:33:21.309698 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.257918 1856 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.317690 1347 consensus_queue.cc:1225] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:21.309360 1663 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:21.263693 1800 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.365012 1854 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.354840 1347 consensus_queue.cc:1046] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:21.356156 1936 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:21.361753 32139 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.414994 1901 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:21.415550 1949 raft_consensus.cc:3032] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
W20260809 07:33:21.416035 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 108528us
W20260809 07:33:21.419054 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 109441us
W20260809 07:33:21.422307 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 142853us
I20260809 07:33:21.427981 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.454007 31006 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.428472 1663 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.453977 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:21.438738 1566 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:21.466640 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.444685 1946 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:21.474967 1724 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:21.475498 1724 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:21.475741 1800 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.476302 31006 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.476992 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.457970 1566 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:21.453557 1949 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:21.490422 1517 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:21.491441 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.496943 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.500746 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.505313 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.458110 1521 consensus_queue.cc:237] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.480990 1621 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:21.490981 1927 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260809 07:33:21.486176 1936 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:21.491047 1566 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:21.527647 1332 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.530382 1934 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:21.530717 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.540761 30900 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:21.489073 1935 raft_consensus.cc:3032] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:21.507651 1854 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.540688 1621 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:21.544813 1856 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.501962 1007 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.543056 1935 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:21.546281 1935 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.547622 1935 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.549974 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.550145 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.550737 30900 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:21.551188 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.551357 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.551587 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.552177 1934 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:21.552330 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.552866 30891 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:21.548264 1817 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.553388 1854 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.534612 1936 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:21.555456 1854 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.525112 1281 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.555554 1936 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.556365 1281 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.557580 1663 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.558375 1281 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.559000 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.559381 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.559574 30911 raft_consensus.cc:2362] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7. Re-sending same reply.
I20260809 07:33:21.556685 1936 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:21.537256 1923 log.cc:924] Time spent T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.437s user 0.017s sys 0.012s
I20260809 07:33:21.552796 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.539731 1497 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.562021 30904 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:21.519202 1724 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.561012 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.583412 31169 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
W20260809 07:33:21.586138 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 109919us
I20260809 07:33:21.587055 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.586493 1856 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.590178 1856 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.590736 1809 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 07:33:21.591747 31005 leader_election.cc:334] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.681s (SENT)
I20260809 07:33:21.592173 31005 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:21.592860 1809 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:21.593286 31005 leader_election.cc:334] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.707s (SENT)
I20260809 07:33:21.594828 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.598492 1946 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:21.599005 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.599620 30896 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.599429 1946 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:21.600693 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 137782us
I20260809 07:33:21.601215 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.601902 31057 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:21.586470 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1254) took 1026 ms. Trace:
I20260809 07:33:21.605371 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1243) took 1164 ms. Trace:
I20260809 07:33:21.605543 31057 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:21.605934 31057 pending_rounds.cc:76] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:21.606431 31057 pending_rounds.cc:97] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 07:33:21.606659 31057 raft_consensus.cc:2866] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:21.608522 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.608860 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.609599 31045 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:21.604316 31049 rpcz_store.cc:276] 0809 07:33:20.559621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:20.861274 (+301653us) service_pool.cc:227] Handling call
0809 07:33:20.866555 (+ 5281us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:20.872667 (+ 6112us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:21.159042 (+286375us) log.cc:842] Serialized 14 byte log entry
0809 07:33:21.586100 (+427058us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:21.586159 (+ 59us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:21.586202 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:21.586268 (+ 66us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:21.586413 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1527,"raft.queue_time_us":131003,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":425094,"threads_started":2,"wal-append.queue_time_us":320156}
I20260809 07:33:21.610658 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.611076 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.611917 1970 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.612622 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.612388 1970 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.613457 31055 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:21.614482 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.614552 1970 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.614951 1838 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.615072 31055 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:21.616143 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.616343 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.617931 31045 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:21.618922 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.619638 31054 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 07:33:21.619547 31057 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:21.620956 30852 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.621729 1800 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:21.622078 1800 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:21.623330 31048 raft_consensus.cc:1226] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:21.639732 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.625486 31046 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: [5.4-6.5]
I20260809 07:33:21.641428 31045 raft_consensus.cc:2379] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:21.639793 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.543676 1544 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.593492 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:21.659065 31175 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:21.659575 30891 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:21.660216 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:21.660565 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.629747 31059 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:21.631026 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:21.629673 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:21.664481 31170 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:21.632058 1822 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:21.633847 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.687045 31055 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:21.634935 30904 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:21.688259 30904 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:21.689218 30904 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.690191 30904 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:21.635075 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.695499 31175 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:21.695804 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.696642 30898 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:21.697410 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.700088 1822 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.638644 30906 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:21.704715 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.705410 30906 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:21.707208 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.640187 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.711189 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.711890 30900 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:21.712576 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.713588 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.525704 1926 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.644325 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1261) took 1040 ms. Trace:
I20260809 07:33:21.644069 1497 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.644834 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.714869 30899 raft_consensus.cc:2362] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6. Re-sending same reply.
I20260809 07:33:21.715202 31181 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:21.715801 1926 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.562428 1895 raft_consensus.cc:686] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:21.716588 1497 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.714555 31046 rpcz_store.cc:276] 0809 07:33:20.746300 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:21.044831 (+298531us) service_pool.cc:227] Handling call
0809 07:33:21.613345 (+568514us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b5800004598. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:33:21.625223 (+ 11878us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:21.644036 (+ 18813us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:21.644258 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":18955520}
I20260809 07:33:21.716990 1895 consensus_queue.cc:237] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.561180 1559 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:21.722864 30890 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:21.536757 1901 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.625548 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:21.537307 1195 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.537401 1898 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.547811 1949 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.666064 1544 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.631835 1856 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.553122 1934 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.671627 1332 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.674535 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1156) took 1201 ms. Trace:
I20260809 07:33:21.643474 1800 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:21.676224 1663 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:21.677152 1855 raft_consensus.cc:3032] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:21.678673 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.662467 1517 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.545104 1621 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.641340 1817 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:21.788772 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:21.684294 1925 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.561s user 0.000s sys 0.002s
I20260809 07:33:21.707986 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:21.605743 31051 rpcz_store.cc:276] 0809 07:33:20.440538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:20.440671 (+ 133us) service_pool.cc:227] Handling call
0809 07:33:20.465574 (+ 24903us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:21.182834 (+717260us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:21.605136 (+422302us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540031cc54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:21.605311 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1042391424}
W20260809 07:33:21.724707 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 59082.3us
I20260809 07:33:21.796558 31006 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.772543 30906 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:21.773497 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.773132 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.797833 31177 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:21.772266 31058 rpcz_store.cc:276] 0809 07:33:20.473314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:20.483849 (+ 10535us) service_pool.cc:227] Handling call
0809 07:33:20.484147 (+ 298us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:20.484508 (+ 361us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:20.484536 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:20.792091 (+307555us) log.cc:842] Serialized 52 byte log entry
0809 07:33:20.854568 (+ 62477us) raft_consensus.cc:1631] Marking committed up to 2
0809 07:33:20.854606 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:20.854669 (+ 63us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:21.515375 (+660706us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:21.666049 (+150674us) raft_consensus.cc:1671] finished
0809 07:33:21.666074 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:21.674457 (+ 8383us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":730,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9240,"spinlock_wait_cycles":108459904,"thread_start_us":62084,"threads_started":1,"wal-append.queue_time_us":197237}
I20260809 07:33:21.798390 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.727223 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.777753 31049 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:21.799000 30911 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:21.789384 30852 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.791419 30904 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260809 07:33:21.796726 30909 raft_consensus.cc:2354] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:21.800665 31164 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:21.782191 1855 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:21.803239 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:21.804235 31054 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:21.787997 1980 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.802563 1331 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:21.808574 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.808933 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.797328 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:21.802183 30904 pending_rounds.cc:76] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Aborting all ops after (but not including) 4
I20260809 07:33:21.809659 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.659090 30896 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:21.806037 1820 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260809 07:33:21.811020 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.811597 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.816313 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.813284 30906 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:21.816461 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.814437 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.813135 30904 pending_rounds.cc:97] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260809 07:33:21.822757 30904 raft_consensus.cc:2866] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:21.822520 30896 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:21.824541 31120 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:21.825889 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.827570 30896 raft_consensus.cc:2354] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:21.821576 30898 raft_consensus.cc:2362] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7. Re-sending same reply.
I20260809 07:33:21.803812 1332 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:21.829411 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.831342 1332 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:21.831753 1332 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.834844 1820 raft_consensus.cc:3032] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:21.835206 1820 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:21.835932 1943 consensus_queue.cc:1046] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:21.816576 1957 consensus_queue.cc:1046] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260809 07:33:21.810895 1958 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:21.837862 1957 consensus_queue.cc:1225] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260809 07:33:21.838768 1972 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:21.835988 1820 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.839677 1820 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.817725 1331 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:21.842406 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:21.838001 1958 raft_consensus.cc:686] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:21.843464 1958 consensus_queue.cc:237] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.838430 1957 raft_consensus.cc:3032] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:21.846820 1957 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:21.850139 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.850579 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.852909 30896 raft_consensus.cc:2362] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4. Re-sending same reply.
I20260809 07:33:21.815857 1902 consensus_queue.cc:1046] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260809 07:33:21.811656 1980 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.847675 1957 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.947597 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1089) took 1010 ms. Trace:
I20260809 07:33:21.935886 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.905697 31181 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 3. Current term is 5. Ops: []
I20260809 07:33:21.966408 31177 rpcz_store.cc:276] 0809 07:33:20.936752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:20.939180 (+ 2428us) service_pool.cc:227] Handling call
0809 07:33:20.996609 (+ 57429us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:21.798244 (+801635us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:21.947337 (+149093us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400184a54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:21.947510 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1840937728}
I20260809 07:33:21.924358 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:21.967823 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 143356us
I20260809 07:33:21.865334 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:21.968633 31120 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:21.864514 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.970959 30900 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:21.863806 1724 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260809 07:33:21.845125 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:21.864215 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.922088 31165 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:21.926479 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:21.967218 30899 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:21.850963 1331 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:21.891642 1347 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.891602 1959 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:22.023351 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 127922us
W20260809 07:33:21.951309 1985 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260809 07:33:21.994103 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.051142 30896 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:21.863782 1822 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:21.892872 1963 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:21.983893 1663 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:22.056325 1963 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.972759 1332 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.926544 1544 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.031219 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:22.059175 1963 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.033267 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:21.884006 1927 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.150s
I20260809 07:33:21.855412 1566 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.906703 1855 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:21.929620 1949 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:22.061745 1566 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.060163 30890 raft_consensus.cc:2396] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260809 07:33:21.884505 1926 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:22.063400 1926 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:22.063525 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.064214 30890 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:22.035247 1195 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.064349 1926 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.059347 30898 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:22.062590 1663 raft_consensus.cc:686] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:21.986331 1817 consensus_queue.cc:1046] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260809 07:33:22.067606 1663 consensus_queue.cc:237] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.033339 31181 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:22.065987 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:21.851130 1856 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:22.060036 1927 consensus_queue.cc:1225] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:22.068140 1817 raft_consensus.cc:3032] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:22.456302 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.415222 1822 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.462414 1817 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:22.462492 1822 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.463533 1817 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.466203 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1091) took 1461 ms. Trace:
I20260809 07:33:22.466344 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:22.472725 30890 raft_consensus.cc:2362] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6. Re-sending same reply.
I20260809 07:33:22.472128 31174 rpcz_store.cc:276] 0809 07:33:21.004958 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:21.005175 (+ 217us) service_pool.cc:227] Handling call
0809 07:33:21.612652 (+607477us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:33:21.612836 (+ 184us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:21.663444 (+ 50608us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:21.663475 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:21.663892 (+ 417us) log.cc:842] Serialized 26 byte log entry
0809 07:33:21.884827 (+220935us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:21.884872 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:21.886607 (+ 1735us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:22.465928 (+579321us) raft_consensus.cc:1671] finished
0809 07:33:22.465947 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:22.466149 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":74824,"raft.queue_time_us":51990,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"spinlock_wait_cycles":27053824,"thread_start_us":177311,"threads_started":1,"wal-append.queue_time_us":354336}
I20260809 07:33:22.473757 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.474464 30890 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:22.475171 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.476164 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.476308 1935 raft_consensus.cc:3032] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:22.476771 1935 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:22.477193 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.477931 30890 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:22.472291 1817 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.481194 1949 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:22.482039 1949 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.037566 1898 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.447882 1331 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.041465 1786 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.450s
I20260809 07:33:22.492115 1331 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.040951 1347 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.059979 31169 raft_consensus.cc:1226] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:22.470434 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:22.072583 32139 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.596s user 0.020s sys 0.012s
W20260809 07:33:22.124432 31005 leader_election.cc:334] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.725s (SENT)
I20260809 07:33:22.471549 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.477658 1935 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.041137 1959 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.458524 30898 raft_consensus.cc:2362] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7. Re-sending same reply.
I20260809 07:33:22.517506 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.458225 30900 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:22.519142 30896 raft_consensus.cc:2379] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:22.505688 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 49473.4us
I20260809 07:33:22.522173 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:22.523160 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.519084 31046 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.528294 30900 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:22.528555 30896 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:22.529795 30896 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260809 07:33:22.495260 1958 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s user 0.044s sys 0.085s
W20260809 07:33:22.517694 1800 consensus_meta.cc:287] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.727s user 0.007s sys 0.012s
I20260809 07:33:22.521917 1497 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:22.547713 1497 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:22.548053 1898 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:22.531330 1902 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:22.548599 1497 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:22.549604 1988 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.280s user 0.006s sys 0.003s
I20260809 07:33:22.551154 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.552733 30904 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:22.554852 30904 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:22.542923 30896 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:22.555634 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.556950 30898 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:22.520907 1521 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.493944 1347 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.505334 1332 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:22.573673 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 449263us
I20260809 07:33:22.577128 30890 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:22.531782 1935 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:22.583696 31005 leader_election.cc:334] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.843s (SENT)
I20260809 07:33:22.584291 31005 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:22.567682 1989 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.347s user 0.000s sys 0.001s
I20260809 07:33:22.565297 1898 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:22.504962 1856 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:22.568044 31046 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:22.565094 1800 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:22.618327 1934 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s user 0.018s sys 0.039s
I20260809 07:33:22.572974 1521 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.742401 1927 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:22.743047 1927 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.620532 1898 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:22.606220 1949 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:22.749027 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 165358us
W20260809 07:33:22.770897 31005 leader_election.cc:334] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.906s (SENT)
I20260809 07:33:22.771317 31005 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:22.773491 1954 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:22.773948 1822 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.676206 1958 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:22.775848 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:22.746833 32139 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.271s user 0.029s sys 0.032s
W20260809 07:33:22.748260 1934 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s user 0.021s sys 0.041s
I20260809 07:33:22.745075 1927 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.786012 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.786793 30909 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:22.778898 1949 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.752532 1566 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
W20260809 07:33:22.795416 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 261217us
I20260809 07:33:22.782017 30906 raft_consensus.cc:2379] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 4: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:22.799459 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.805444 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:22.806246 31170 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:22.801918 30852 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.810832 31046 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:22.811205 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.811690 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:22.812220 31059 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:22.793511 1927 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:22.794963 1954 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.835141 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.784981 1497 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.815599 2000 raft_consensus.cc:2781] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:22.842075 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:22.842667 31046 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:22.842587 1954 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:22.843708 1995 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.066s user 0.000s sys 0.001s
I20260809 07:33:22.816111 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:22.844434 31054 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.844913 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.845497 30890 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:22.817956 1506 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:22.846683 1506 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:22.847071 1506 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:22.842409 1927 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.811896 30904 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:22.819509 31172 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:22.850723 1927 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.820328 1800 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.807745 1959 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:22.797816 32139 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 1.4902e+06us
I20260809 07:33:22.821266 1898 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.821928 1521 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.822995 31051 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:22.799983 1949 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.825529 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:22.825635 30741 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40564 (request call id 149) took 13269 ms (client timeout 15000 ms). Trace:
I20260809 07:33:22.860301 30900 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.826365 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.826507 31170 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:22.826550 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.827772 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:22.836709 31049 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:22.861862 31175 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:22.837961 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:22.862797 31059 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:22.840673 30744 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: term changed from 4 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:22.801017 2005 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:22.854123 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:22.864269 30904 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.861953 31058 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:22.841957 1497 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.858801 31054 raft_consensus.cc:2454] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:22.848300 1954 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.859380 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.870361 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.875108 2005 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:22.876186 1949 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.877022 30744 catalog_manager.cc:5495] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: term changed from 5 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: NON_VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: true } } }
I20260809 07:33:22.857928 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:22.806600 1822 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260809 07:33:22.806360 1934 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260809 07:33:22.879633 30896 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.881579 30904 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:22.883678 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.887627 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.888856 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:22.890527 30904 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:22.877791 2005 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.905886 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.906680 31172 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:22.901105 30900 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:22.909416 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.911197 30852 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.910441 30900 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:22.860242 30741 rpcz_store.cc:269] 0809 07:33:09.560250 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:09.582889 (+ 22639us) service_pool.cc:227] Handling call
0809 07:33:22.825566 (+13242677us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:19.645780 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:19.686100 (+ 40320us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:19.686148 (+ 48us) write_op.cc:435] Acquired schema lock
0809 07:33:19.686160 (+ 12us) tablet.cc:660] Decoding operations
0809 07:33:19.731296 (+ 45136us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:19.731410 (+ 114us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:19.731454 (+ 44us) tablet.cc:683] Acquiring locks for 13 operations
0809 07:33:19.732650 (+ 1196us) tablet.cc:699] Row locks acquired
0809 07:33:19.732680 (+ 30us) write_op.cc:260] PREPARE: finished
0809 07:33:19.732834 (+ 154us) write_op.cc:270] Start()
0809 07:33:19.732985 (+ 151us) write_op.cc:276] Timestamp: P: 1786260799732775 usec, L: 0
0809 07:33:19.733017 (+ 32us) op_driver.cc:331] REPLICATION: starting
0809 07:33:19.733795 (+ 778us) log.cc:842] Serialized 5207 byte log entry
0809 07:33:20.279597 (+545802us) op_driver.cc:444] REPLICATION: finished
0809 07:33:20.394244 (+114647us) write_op.cc:301] APPLY: starting
0809 07:33:20.424580 (+ 30336us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:20.452220 (+ 27640us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:20.452361 (+ 141us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:20.455671 (+ 3310us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:20.455960 (+ 289us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0809 07:33:20.456030 (+ 70us) write_op.cc:312] APPLY: finished
0809 07:33:20.456402 (+ 372us) log.cc:842] Serialized 94 byte log entry
0809 07:33:20.456578 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:20.456717 (+ 139us) write_op.cc:454] Released schema lock
0809 07:33:20.456788 (+ 71us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:20.485163 (+ 28375us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:20.488398 (+ 3235us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":123223,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":96295,"apply.run_cpu_time_us":20260,"apply.run_wall_time_us":427373,"num_ops":13,"prepare.queue_time_us":270985,"prepare.run_cpu_time_us":14001,"prepare.run_wall_time_us":163433,"raft.queue_time_us":464593,"raft.run_cpu_time_us":6549,"raft.run_wall_time_us":18530,"replication_time_us":546469,"thread_start_us":566488,"threads_started":4,"wal-append.queue_time_us":74787,"wal-append.run_cpu_time_us":21549,"wal-append.run_wall_time_us":1649153}]]}
I20260809 07:33:22.917794 30904 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:22.921490 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.921905 30896 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:22.922708 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.930770 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.930137 31059 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:22.931638 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 970) took 1090 ms. Trace:
I20260809 07:33:22.932060 30911 rpcz_store.cc:276] 0809 07:33:21.841468 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:22.930474 (+1089006us) service_pool.cc:227] Handling call
0809 07:33:22.931579 (+ 1105us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:22.932739 2001 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260809 07:33:22.934815 30911 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:22.936124 30911 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:22.937382 30911 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:22.938668 30911 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:22.949415 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:22.904373 1962 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:22.954710 2011 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:22.951680 2000 raft_consensus.cc:686] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:22.955322 2011 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:22.955750 2000 consensus_queue.cc:237] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.957755 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:22.958437 31059 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 6.
I20260809 07:33:22.956221 1855 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.960095 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:22.960467 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:22.961426 30900 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:22.962879 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:22.963338 1521 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.952272 1901 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.963815 1521 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.962090 31005 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:22.902907 1954 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
W20260809 07:33:22.970952 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 59854.3us
I20260809 07:33:22.903990 1958 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260809 07:33:22.972203 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.967846 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:22.974517 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:22.957651 1332 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:22.957839 1962 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:22.976194 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:22.959839 2009 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.957443 1856 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:22.966944 1935 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:22.964602 1901 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.981841 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:22.981585 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:22.970938 1822 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:22.970870 1895 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:22.989643 2011 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:22.986372 1195 raft_consensus.cc:3032] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:22.990674 1195 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:22.983633 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.980195 1926 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260809 07:33:22.985477 1980 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.986454 1566 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:22.982460 1663 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:22.993593 1663 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:22.994007 1663 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.986928 1901 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.987112 1820 raft_consensus.cc:3032] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:22.994556 1820 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:22.992286 1935 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.995711 31182 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:22.995360 1820 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:22.996236 1935 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:22.979803 1332 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:22.996795 1332 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.998471 31175 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:22.998692 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.999603 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.999861 31177 raft_consensus.cc:1226] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.001677 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.001888 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.002449 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.002382 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.003212 31172 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.003402 31174 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:22.987677 1800 raft_consensus.cc:2741] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 07:33:23.004773 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.999859 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.005407 31182 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:23.002678 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:22.987907 1521 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:22.977360 2013 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:22.977406 1991 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260809 07:33:22.980281 1962 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:22.988379 30853 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:22.977255 1786 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:22.996376 1820 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:22.987551 30852 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.004042 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1086) took 1078 ms. Trace:
W20260809 07:33:23.009622 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 47605.7us
I20260809 07:33:23.010228 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.010270 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.017894 31175 raft_consensus.cc:2379] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:23.013680 1898 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.013929 31182 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.014616 31181 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:23.016054 31177 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.015622 1827 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:23.019856 1957 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.020112 31181 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:23.020241 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.020916 31175 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:23.021060 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 966) took 1200 ms. Trace:
I20260809 07:33:23.021413 30906 rpcz_store.cc:276] 0809 07:33:21.820436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:21.829147 (+ 8711us) service_pool.cc:227] Handling call
0809 07:33:23.020845 (+1191698us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54001a40d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:23.020982 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2117208448}
I20260809 07:33:23.022572 30898 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:23.015890 2019 consensus_queue.cc:1046] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260809 07:33:23.026686 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.027541 30899 raft_consensus.cc:2396] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:23.027944 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 950) took 1379 ms. Trace:
I20260809 07:33:23.028230 30899 rpcz_store.cc:276] 0809 07:33:21.648997 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:23.015494 (+1366497us) service_pool.cc:227] Handling call
0809 07:33:23.027890 (+ 12396us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:23.011467 2009 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.032002 31170 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.032228 31175 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.032517 1506 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.017967 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:23.033515 31174 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.033926 31181 raft_consensus.cc:2932] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Committing config change with OpId 6.5: config changed from index 2 to 5, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
W20260809 07:33:23.036329 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 48848.9us
I20260809 07:33:23.039243 30852 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.036561 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.017431 31164 rpcz_store.cc:276] 0809 07:33:21.926004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:21.926142 (+ 138us) service_pool.cc:227] Handling call
0809 07:33:23.003865 (+1077723us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54000320d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:23.003988 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1972660096}
I20260809 07:33:23.037377 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.016553 31172 raft_consensus.cc:1226] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.040438 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.040380 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.043013 31177 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:33:23.044553 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 56253us
I20260809 07:33:23.046339 31175 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.047488 31175 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.046851 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.047592 31177 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:23.050961 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.047493 31172 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:23.051637 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.054211 31059 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.051690 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.052452 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.052418 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.054972 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.055006 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.056385 30896 raft_consensus.cc:3037] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.050587 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.045763 1936 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:23.058954 31165 raft_consensus.cc:2354] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:23.059051 1936 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.058723 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.048235 1895 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:23.060398 1895 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:23.062163 30900 raft_consensus.cc:1261] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:23.063001 30906 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 5: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:23.063800 31051 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.040885 1517 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.055869 31058 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:23.032109 2009 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.066555 30898 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:23.038491 1935 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:23.070467 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:23.073261 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 55385.5us
I20260809 07:33:23.075103 31051 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.078217 30896 raft_consensus.cc:2454] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:23.080480 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.080490 1943 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:23.081085 30852 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.081228 1943 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.082021 2023 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.041392 2000 raft_consensus.cc:2781] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:23.090008 2023 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:23.090199 2000 raft_consensus.cc:686] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.091321 2000 consensus_queue.cc:237] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.092752 2023 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.083354 1943 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.083882 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.093739 1800 raft_consensus.cc:3032] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:23.084619 30852 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.094129 1800 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.094466 2006 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:23.094801 30852 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.085364 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.043285 1991 consensus_queue.cc:1225] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:23.086823 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.081279 30899 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:23.108579 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.108834 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.055069 1898 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:23.113045 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.113951 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.117808 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.073042 2013 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:23.126928 31164 raft_consensus.cc:2354] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7.
I20260809 07:33:23.042268 1827 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.108047 1943 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:23.142937 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.148442 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.104588 1935 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.120141 2019 consensus_queue.cc:1046] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260809 07:33:23.152889 1521 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:23.153313 2019 consensus_queue.cc:1225] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260809 07:33:23.153909 2019 raft_consensus.cc:3032] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:23.154241 2019 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.155093 2019 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.095013 1800 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.121276 1994 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.158912 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.163125 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.165748 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.166378 31174 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:23.166697 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.169965 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.105967 1786 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
I20260809 07:33:23.170687 31172 raft_consensus.cc:3037] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.169508 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.174715 2011 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:23.171532 1898 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:23.175279 1506 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.147203 1943 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.176388 1943 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.177222 1663 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.165818 1994 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.178650 1994 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.153338 1521 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.182511 1800 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.184969 1195 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.187599 2006 consensus_queue.cc:1225] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:23.188638 1962 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.189209 30891 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:23.189700 1949 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:23.118113 1517 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260809 07:33:23.197578 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:23.198552 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 106173us
I20260809 07:33:23.131095 1901 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260809 07:33:23.164964 2019 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.186195 1195 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.207554 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:23.201272 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 114517us
I20260809 07:33:23.200444 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:23.203737 30891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 958) took 1472 ms (client timeout 1748 ms). Trace:
I20260809 07:33:23.166661 2013 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:23.167527 1817 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260809 07:33:23.212082 30891 rpcz_store.cc:269] 0809 07:33:21.731104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:21.798540 (+ 67436us) service_pool.cc:227] Handling call
0809 07:33:23.203579 (+1405039us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54001f1154. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:23.203684 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3102536960}
I20260809 07:33:23.190187 1949 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.196779 1521 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:23.218062 1506 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:23.221351 1926 raft_consensus.cc:3032] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:23.221495 1506 raft_consensus.cc:686] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.221889 1926 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.199118 2011 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.214790 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.203359 1822 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.214411 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.224445 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.214412 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:23.214908 31172 raft_consensus.cc:2454] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.144407 1895 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:23.262475 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48161.3us
I20260809 07:33:23.263280 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.150692 1936 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.222792 1926 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.215492 2013 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.230796 1827 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.208928 1898 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.231423 1949 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.231851 2011 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.227174 1822 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:23.237797 1506 consensus_queue.cc:237] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.236297 2023 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.238467 1332 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.206765 1994 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.240056 1663 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.240515 2031 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:23.236361 30911 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:23.245577 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.246210 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.247159 2033 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:23.298123 2033 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:23.248109 2009 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:23.248787 31170 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.249027 1962 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.249328 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.246140 31164 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:23.249646 1901 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:23.248035 31165 raft_consensus.cc:1261] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260809 07:33:23.249711 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.250671 1517 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:23.251626 1980 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:23.252705 1621 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:23.319476 1332 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.299192 2033 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.255333 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.321039 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.251266 1943 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.256775 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.256850 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.258028 30853 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.322786 31177 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.249652 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.323465 1332 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.266808 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:23.323208 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 65292.5us
I20260809 07:33:23.324211 31049 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.266927 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.325035 31172 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:23.325344 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.326130 1994 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.326716 1994 consensus_queue.cc:1046] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:23.326872 30853 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.328945 1943 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.329120 1195 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:23.329463 1663 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:23.329835 1663 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.271162 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:23.330471 1800 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:23.330929 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 59850.5us
I20260809 07:33:23.331640 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.310289 1926 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.331670 30853 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.259958 1521 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.271880 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:23.273319 1898 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.336586 1962 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.338091 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.338670 1962 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.338843 2006 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:23.339567 31054 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.274695 2032 raft_consensus.cc:2741] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 07:33:23.275262 1820 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.275756 1544 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:23.270107 2013 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:23.284698 1822 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:23.309077 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 95412.9us
I20260809 07:33:23.315070 31051 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:23.314545 31165 pending_rounds.cc:76] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:23.321967 31058 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:23.322954 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 66208.8us
I20260809 07:33:23.328961 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:23.333386 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.333806 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.330129 1663 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.351439 31165 pending_rounds.cc:97] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260809 07:33:23.353230 31049 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.353711 31165 raft_consensus.cc:2866] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:23.354592 30853 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.355840 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.360189 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.360530 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.360867 30896 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.361380 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.361976 30898 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:23.363809 31049 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:23.335687 31172 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:23.335752 30852 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.338233 1946 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:23.369267 1621 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:23.369818 1946 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:23.311041 2009 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.344017 2033 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.370275 2023 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.345009 1544 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.372987 1544 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.353114 31054 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.373607 30896 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:23.374768 2023 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.374584 2009 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.375514 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.375677 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.376399 30852 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.376832 30853 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.377184 1949 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.378098 31005 leader_election.cc:398] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:23.378476 31005 leader_election.cc:401] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:23.379042 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.379717 31054 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.367153 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.383069 30911 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:23.367411 1994 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.367828 1943 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.368044 2032 consensus_queue.cc:1035] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:23.368456 30909 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:23.368557 30899 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.367828 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.392365 30891 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:23.383215 30896 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:23.377619 1949 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.343164 1962 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.394349 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.370218 1946 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:23.396499 31058 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:23.395937 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.398488 30853 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.407604 30853 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.404112 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.408294 30896 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:23.339726 1521 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.402251 31054 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:23.404685 30891 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:23.390883 1332 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.390802 1957 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:23.417143 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.417761 30909 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:23.408633 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:23.396010 1856 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:23.403200 1949 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.410230 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.411164 30852 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.421461 30896 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:23.408691 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:23.422731 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 85031.2us
I20260809 07:33:23.423638 1517 raft_consensus.cc:3032] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:23.427049 2044 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:23.406630 1800 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:23.435141 30853 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.431442 31051 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:23.433764 1621 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:23.481956 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.482193 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:23.506851 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 97437.4us
I20260809 07:33:23.510725 30852 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.407243 1994 consensus_queue.cc:1046] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260809 07:33:23.481313 31054 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
W20260809 07:33:23.481185 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46122.5us
I20260809 07:33:23.481804 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.412303 1855 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:23.404992 2000 consensus_queue.cc:1046] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260809 07:33:23.412482 1943 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:23.412969 1827 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
W20260809 07:33:23.479447 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48823.1us
I20260809 07:33:23.586721 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:23.586977 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.506989 31059 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.506577 31172 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260809 07:33:23.508126 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 99563.1us
I20260809 07:33:23.516095 31046 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:23.590478 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.519019 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.591481 31051 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:23.410641 1195 raft_consensus.cc:3032] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:23.519929 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.550094 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.551003 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.596494 31055 raft_consensus.cc:2379] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:23.551173 31181 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.432144 1957 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.432628 1517 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.550202 31058 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:23.520749 31049 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:23.600224 31051 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:23.482206 2023 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.458923 1521 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:23.556599 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 45964.8us
I20260809 07:33:23.603575 30852 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:23.556773 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.552337 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.604825 31054 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 5
I20260809 07:33:23.509508 1927 raft_consensus.cc:2741] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 07:33:23.550209 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.515717 1621 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.458287 32141 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.610622 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.432896 1820 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260809 07:33:23.506541 30906 pending_rounds.cc:76] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Aborting all ops after (but not including) 4
I20260809 07:33:23.523743 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.599835 31046 raft_consensus.cc:1226] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:23.616523 30898 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:23.601289 31058 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.560442 1800 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.595798 1195 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.513407 1991 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:23.601770 1347 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:23.617718 30906 pending_rounds.cc:97] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260809 07:33:23.627310 30906 raft_consensus.cc:2866] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:23.628855 30898 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:23.605523 2044 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:23.619539 31005 leader_election.cc:398] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:23.621025 31058 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:23.630069 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.604259 1994 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:23.633176 31005 leader_election.cc:401] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260809 07:33:23.635636 1827 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:23.636097 1827 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.640535 2031 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:23.641410 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.642112 31058 raft_consensus.cc:2379] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:23.615562 1935 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:23.644083 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.614670 1621 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:23.650504 31045 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:23.650542 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.651553 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.652141 31005 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.653420 2031 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.655176 1800 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.656920 31059 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:23.658706 31048 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.609099 1521 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.649271 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.661623 1521 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.661470 1822 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.663359 1822 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.663821 31177 raft_consensus.cc:1226] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.646412 1347 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.665098 1347 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:23.666486 2052 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.137s user 0.001s sys 0.000s
I20260809 07:33:23.644155 1827 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:23.667764 1957 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.674561 30909 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:23.618896 31054 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:23.675712 31057 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:23.679401 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.679554 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.680630 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.681085 31057 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.681452 31054 raft_consensus.cc:2379] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:23.681290 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.682265 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.682888 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:23.678797 1195 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.657999 1935 raft_consensus.cc:686] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.696229 1901 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.684264 2031 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:23.671422 1946 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.684911 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.644618 1991 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.687110 31005 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:23.684900 2053 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260809 07:33:23.683741 31058 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:23.756307 31058 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:23.757602 31058 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:23.700124 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.658305 2044 raft_consensus.cc:686] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:23.720013 2054 log.cc:924] Time spent T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.078s user 0.001s sys 0.000s
I20260809 07:33:23.716729 31057 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:23.721931 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.730336 31054 raft_consensus.cc:1261] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260809 07:33:23.731323 1936 raft_consensus.cc:3032] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:23.736563 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:23.752061 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 54245.2us
I20260809 07:33:23.766966 30890 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:23.740139 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:23.754683 30904 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:23.722865 1901 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:23.762136 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 211201us
I20260809 07:33:23.770809 30853 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:23.766161 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 44301us
I20260809 07:33:23.768613 31049 raft_consensus.cc:1226] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:23.768016 30899 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
W20260809 07:33:23.769172 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 82151.1us
I20260809 07:33:23.773345 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:23.652429 1517 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.775663 31120 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.777021 30853 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.766721 1936 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:23.782855 30853 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.784250 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:23.784801 30890 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.786691 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.790952 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.762565 2023 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:23.751295 1566 consensus_queue.cc:1046] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260809 07:33:23.796510 30896 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:23.765848 1980 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:23.768939 2006 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.775079 1901 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.791332 2019 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:23.810632 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:23.812086 31005 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.814256 31057 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:23.791826 2009 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.790485 1927 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:23.820134 2033 consensus_queue.cc:1046] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:23.821902 31005 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:23.822468 2006 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:23.822795 30853 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:23.823232 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 46333.8us
I20260809 07:33:23.829571 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:23.832901 30890 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:23.803287 1959 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:23.851758 1959 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:23.852133 1959 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:23.833595 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:23.811295 1957 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.783792 2013 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.825788 1529 raft_consensus.cc:3032] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:23.793941 1822 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260809 07:33:23.859824 30853 leader_election.cc:398] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:23.860555 1544 consensus_queue.cc:1046] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:23.861189 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.824004 1663 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.870536 30853 leader_election.cc:401] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:23.803299 2031 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260809 07:33:23.824637 2062 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:23.806491 2023 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:23.824007 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 395259us
I20260809 07:33:23.877415 2006 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:23.798378 1935 consensus_queue.cc:237] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.873646 2070 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:23.943990 2025 log.cc:924] Time spent T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260809 07:33:23.858592 1927 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260809 07:33:23.803107 1750 consensus_queue.cc:1046] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
W20260809 07:33:23.937469 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 96090.3us
I20260809 07:33:23.868727 1529 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:23.874931 1934 raft_consensus.cc:3032] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:23.861707 1347 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:23.943583 31181 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260809 07:33:23.790910 1521 consensus_queue.cc:1159] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a720817fc113446a8ed55e61523b280a" responder_term: 4 status { last_received { term: 3 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260809 07:33:23.938706 1980 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:23.953490 1663 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:23.917765 2067 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:23.824517 1991 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:23.953934 1962 consensus_meta.cc:287] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.525s user 0.006s sys 0.000s
I20260809 07:33:23.963905 1946 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:23.965950 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 142848us
I20260809 07:33:23.873485 2011 consensus_queue.cc:1046] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
W20260809 07:33:23.968806 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 93938.4us
I20260809 07:33:23.977116 30853 leader_election.cc:398] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7.
I20260809 07:33:24.017968 30853 leader_election.cc:401] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:23.839618 1936 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:23.857095 2044 consensus_queue.cc:237] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:23.983793 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 161976us
I20260809 07:33:24.019477 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:24.023164 2023 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.024231 1621 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.027325 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a
I20260809 07:33:23.979903 1934 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:24.027980 2077 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
W20260809 07:33:23.985788 1949 consensus_meta.cc:287] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.527s user 0.008s sys 0.002s
W20260809 07:33:24.029405 2075 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260809 07:33:24.029536 1949 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.030856 2076 log.cc:924] Time spent T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260809 07:33:24.031708 1949 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.032073 31169 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260809 07:33:24.034889 2087 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260809 07:33:24.035979 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:24.036221 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.036767 31049 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.040063 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.013188 1962 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.042804 2013 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:24.043151 1962 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.043741 31049 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:23.968708 2078 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260809 07:33:23.979775 2032 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:24.045656 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.023265 2011 raft_consensus.cc:3032] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:24.047113 2011 raft_consensus.cc:729] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.026258 1621 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:24.047948 2011 consensus_queue.cc:260] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.048592 1529 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.049177 2011 raft_consensus.cc:3037] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.049746 1529 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:24.051366 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 74385us
I20260809 07:33:24.047837 30852 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:24.030150 2072 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260809 07:33:24.052615 30853 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.051245 2023 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7.
I20260809 07:33:24.055258 1943 raft_consensus.cc:3032] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:24.055630 1943 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:24.056375 1943 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.057677 1943 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.059782 1517 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:24.048424 1901 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.063118 1901 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:24.065469 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46060.5us
I20260809 07:33:24.066205 31005 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.068516 1195 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:24.028606 1934 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.070237 1934 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.070945 30900 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:24.071559 30900 raft_consensus.cc:595] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.078086 1958 raft_consensus.cc:3032] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:24.078516 1958 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.079275 1958 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.080430 1958 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.088445 1936 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.054117 1800 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:24.090899 1800 raft_consensus.cc:686] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:24.050673 1959 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.092734 31172 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:24.093417 31172 raft_consensus.cc:595] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.093732 31172 raft_consensus.cc:603] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:24.094385 1959 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.094861 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 24) took 8574 ms. Trace:
I20260809 07:33:24.091830 1800 consensus_queue.cc:237] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.086099 2074 log.cc:924] Time spent T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.144s user 0.000s sys 0.001s
I20260809 07:33:24.101279 31172 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:24.101898 31172 raft_consensus.cc:595] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.054152 1855 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.095177 30744 rpcz_store.cc:276] 0809 07:33:15.520306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:15.520488 (+ 182us) service_pool.cc:227] Handling call
0809 07:33:24.094800 (+8574312us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:22.979577 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:22.989885 (+ 10308us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:22.989912 (+ 27us) write_op.cc:435] Acquired schema lock
0809 07:33:22.989922 (+ 10us) tablet.cc:660] Decoding operations
0809 07:33:22.990247 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:22.990344 (+ 97us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:22.990372 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0809 07:33:22.990612 (+ 240us) tablet.cc:699] Row locks acquired
0809 07:33:22.990626 (+ 14us) write_op.cc:260] PREPARE: finished
0809 07:33:22.990766 (+ 140us) write_op.cc:270] Start()
0809 07:33:22.990881 (+ 115us) write_op.cc:276] Timestamp: P: 1786260802990720 usec, L: 0
0809 07:33:22.990901 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 07:33:22.991539 (+ 638us) log.cc:842] Serialized 1032 byte log entry
0809 07:33:23.269171 (+277632us) op_driver.cc:444] REPLICATION: finished
0809 07:33:23.508625 (+239454us) write_op.cc:301] APPLY: starting
0809 07:33:23.515575 (+ 6950us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:23.556491 (+ 40916us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:23.582631 (+ 26140us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:23.852101 (+269470us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5c00155b0c. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409b282501 00007f409b2823f1 00007f409b27d12c 00007f409882391b 00007f40988235e3 00007f409891db14 00007f4098909dfa 00007f409890ddd1 00007f409890dd89 00007f409890dd19 00007f409890dce1
0809 07:33:23.852482 (+ 381us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:23.852647 (+ 165us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0809 07:33:23.852687 (+ 40us) write_op.cc:312] APPLY: finished
0809 07:33:23.852886 (+ 199us) log.cc:842] Serialized 28 byte log entry
0809 07:33:23.853046 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:23.853135 (+ 89us) write_op.cc:454] Released schema lock
0809 07:33:23.853172 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:23.860119 (+ 6947us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:23.860147 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1934055,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":152448,"apply.run_cpu_time_us":16351,"apply.run_wall_time_us":424518,"num_ops":2,"prepare.queue_time_us":68517,"prepare.run_cpu_time_us":6380,"prepare.run_wall_time_us":36224,"raft.queue_time_us":114238,"raft.run_cpu_time_us":22143,"raft.run_wall_time_us":52790,"replication_time_us":278169,"spinlock_wait_cycles":23108992,"thread_start_us":188796,"threads_started":4,"wal-append.queue_time_us":70781}]]}
I20260809 07:33:24.104341 31055 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:24.113339 1855 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 07:33:24.115495 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 88238.1us
W20260809 07:33:24.118467 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 49981.6us
I20260809 07:33:24.120839 31005 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.126964 30853 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:24.128532 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 59648.7us
I20260809 07:33:24.123289 1663 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.130143 2094 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:24.136386 2096 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 07:33:24.136860 2078 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:24.137274 2078 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.115190 1980 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260809 07:33:24.138034 2078 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.138890 1827 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:24.139503 1855 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.130498 2019 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260809 07:33:24.139257 2078 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.142346 1820 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:24.138607 1980 raft_consensus.cc:3032] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:24.146588 2096 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:24.147053 2096 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.147591 31169 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:24.148192 31169 raft_consensus.cc:595] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.147780 2096 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.148972 2096 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.149634 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.146562 1980 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.143831 2092 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:24.150964 1980 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.151288 1521 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:24.130196 2033 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:24.151847 1506 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:24.152146 2033 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.152547 1926 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:24.152603 1521 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:24.152985 2033 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.155341 31005 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.152264 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:24.151574 2092 raft_consensus.cc:3032] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:24.164974 31005 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.166239 31005 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.167454 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.175026 31165 raft_consensus.cc:2396] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 07:33:24.176975 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.179451 31005 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:24.180656 31005 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.153277 1926 consensus_queue.cc:1225] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 07:33:24.183404 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.184734 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:24.194856 31165 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:24.195437 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.200346 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.200954 30896 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.202598 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.203830 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.204385 31059 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:24.205400 30904 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:24.206530 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.207182 30911 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:24.209447 31051 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:24.210548 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.213169 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.213927 31058 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:24.216614 30891 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:24.198406 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.215621 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:24.185650 30745 catalog_manager.cc:5495] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.218663 31055 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:24.217823 31164 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:24.220273 31164 raft_consensus.cc:595] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.220616 31164 raft_consensus.cc:603] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:24.223994 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.224550 30890 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:24.220592 30745 catalog_manager.cc:5495] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:24.175390 1895 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:24.227274 30745 catalog_manager.cc:5495] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:24.237336 31177 raft_consensus.cc:3032] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:24.239023 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.235469 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.174407 1827 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.178649 2092 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.182989 2006 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.236217 30900 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.236085 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.186544 1959 raft_consensus.cc:2781] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:24.183272 2032 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:24.237934 30896 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:24.232659 31055 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:24.239028 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:33:24.233821 31045 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.276551 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:24.277348 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:24.238161 30745 catalog_manager.cc:5495] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.242918 30911 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:24.182909 1901 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
W20260809 07:33:24.245541 2113 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260809 07:33:24.133615 1935 consensus_queue.cc:1046] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260809 07:33:24.250478 2067 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:24.233942 2094 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:24.251060 2099 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:24.301903 2099 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.302758 2099 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.223351 1980 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.196537 1820 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.249956 30853 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.259438 31058 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260809 07:33:24.239362 31177 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.313117 30890 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 10.
I20260809 07:33:24.313323 31182 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:24.271636 1959 raft_consensus.cc:686] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.283713 30745 catalog_manager.cc:5495] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:24.316676 30745 catalog_manager.cc:5495] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.186976 1855 consensus_queue.cc:1035] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260809 07:33:24.216002 1521 consensus_queue.cc:1046] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260809 07:33:24.256392 2044 raft_consensus.cc:2741] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260809 07:33:24.296418 31045 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:24.298557 2067 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.299494 1936 raft_consensus.cc:3032] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:24.308156 31049 raft_consensus.cc:2396] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260809 07:33:24.326226 1936 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:24.327464 2067 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.327085 1936 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.262358 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:24.314128 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 64235.1us
I20260809 07:33:24.326937 30745 catalog_manager.cc:5495] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 4, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.294162 2098 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:24.324680 31048 raft_consensus.cc:1261] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.314874 31177 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.332828 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 70584.6us
I20260809 07:33:24.334352 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.334764 30853 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.335477 31120 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.285298 2013 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.036s
W20260809 07:33:24.338349 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 156062us
I20260809 07:33:24.318342 1935 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.345494 1820 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:24.347977 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.348420 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.348745 1856 raft_consensus.cc:3032] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:24.349077 30890 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:24.350121 2009 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:24.350704 2009 raft_consensus.cc:686] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.351660 2009 consensus_queue.cc:237] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.353014 2098 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:24.353497 2098 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.363441 30890 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.364308 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.365787 31169 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260809 07:33:24.366166 31169 pending_rounds.cc:76] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:24.366431 31169 pending_rounds.cc:97] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260809 07:33:24.366655 31169 raft_consensus.cc:2866] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:24.359985 1506 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:24.372939 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:24.341005 1926 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:24.377964 31048 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:24.349108 1856 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:24.340111 2078 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:24.339932 1994 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.364993 2061 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260809 07:33:24.410720 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.329388 1962 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260809 07:33:24.378872 2008 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260809 07:33:24.323375 2092 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.412557 31181 raft_consensus.cc:1261] T 3c486ed5f0d9423992400dfce71ce70f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.391678 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.419009 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.382496 30745 catalog_manager.cc:5495] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 5, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.383813 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.383767 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:24.435334 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 202520us
W20260809 07:33:24.436462 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 58493us
I20260809 07:33:24.450120 30896 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:24.452193 1506 raft_consensus.cc:686] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.459692 1506 consensus_queue.cc:237] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.458730 31169 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:24.471312 2078 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:24.472136 2078 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:24.473106 2078 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.479113 30852 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:24.481118 1994 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:24.481521 1994 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:24.481917 1994 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:24.481630 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 115357us
I20260809 07:33:24.484278 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.488391 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.486018 30745 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a reported cstate change: config changed from index 3 to 6, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.411713 1959 consensus_queue.cc:237] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.498159 2112 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.191s user 0.001s sys 0.000s
I20260809 07:33:24.452418 1529 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:24.501031 30745 catalog_manager.cc:5495] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 4, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.513372 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.509773 30745 catalog_manager.cc:5495] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.515027 30745 catalog_manager.cc:5495] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 5, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.516624 30745 catalog_manager.cc:5495] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.519263 31048 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:24.518085 30745 catalog_manager.cc:5495] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.522481 30745 catalog_manager.cc:5495] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 6 to 7, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 7 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.524448 30745 catalog_manager.cc:5495] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.527172 30745 catalog_manager.cc:5495] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 7, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 7 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:24.529170 30745 catalog_manager.cc:5495] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 2 to 4, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 4 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.533586 30900 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260809 07:33:24.520622 1521 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:24.445801 1856 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.515244 2092 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:24.535411 30745 catalog_manager.cc:5495] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.548218 30745 catalog_manager.cc:5495] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.550274 30745 catalog_manager.cc:5495] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 7, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 7 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.553948 30745 catalog_manager.cc:5495] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 3, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 3 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.555558 30745 catalog_manager.cc:5495] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 5 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:24.557881 30745 catalog_manager.cc:5495] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.512512 1529 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.558657 1949 consensus_queue.cc:1159] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a720817fc113446a8ed55e61523b280a" responder_term: 6 status { last_received { term: 6 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 4 } }
I20260809 07:33:24.576932 1529 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:24.553249 2108 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.172s user 0.001s sys 0.000s
I20260809 07:33:24.537474 1195 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260809 07:33:24.564687 2116 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.134s user 0.001s sys 0.000s
I20260809 07:33:24.570420 2006 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.579658 30911 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:24.580575 30906 raft_consensus.cc:1226] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:24.580703 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.581408 30896 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:24.582006 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.574385 2078 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.582561 30896 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:24.583945 1856 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.585245 2078 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:24.537134 1521 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:24.588660 31175 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:24.595341 1663 raft_consensus.cc:3032] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:24.595734 1663 raft_consensus.cc:729] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.590515 31059 raft_consensus.cc:1261] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:24.591231 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.600623 31177 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.602665 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:24.604172 2011 consensus_meta.cc:287] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.555s user 0.009s sys 0.000s
I20260809 07:33:24.583710 2131 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:24.607897 31174 raft_consensus.cc:1261] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260809 07:33:24.620143 31174 pending_rounds.cc:76] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 2
I20260809 07:33:24.620457 31174 pending_rounds.cc:97] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260809 07:33:24.620718 31174 raft_consensus.cc:2866] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260809 07:33:24.622754 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 138640us
I20260809 07:33:24.596542 1521 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:24.634200 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.640950 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:24.641709 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 43930.1us
I20260809 07:33:24.598351 30745 catalog_manager.cc:5495] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:24.647396 30745 catalog_manager.cc:5495] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 5, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 5 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.648487 30745 catalog_manager.cc:5495] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 5 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:24.655109 1943 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260809 07:33:24.626242 2131 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.664474 2131 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.665940 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:24.666772 31164 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:33:24.667145 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 57728.5us
I20260809 07:33:24.644580 2011 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:24.646552 2013 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:24.676397 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.640829 1962 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260809 07:33:24.642007 2019 consensus_queue.cc:1035] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260809 07:33:24.679019 31181 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.684659 2011 raft_consensus.cc:686] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.687958 2078 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:24.688730 2013 raft_consensus.cc:686] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.689621 2013 consensus_queue.cc:237] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.692565 2009 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:24.686108 2011 consensus_queue.cc:237] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.694430 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:24.695194 2009 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:24.695650 2009 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.698618 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:24.698771 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.699622 31120 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:24.701620 1544 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:24.654222 1663 consensus_queue.cc:260] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.703053 1895 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:24.704331 2008 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:24.654384 2135 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260809 07:33:24.705494 2135 raft_consensus.cc:3032] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:24.705109 2008 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.706091 2135 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.707440 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.708019 31059 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:24.653641 2023 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260809 07:33:24.710587 30906 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:24.708513 2078 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:24.711813 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.712047 31170 raft_consensus.cc:2396] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:24.700270 2061 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:24.724365 31164 raft_consensus.cc:1261] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:24.697301 1827 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260809 07:33:24.728222 1856 consensus_queue.cc:1035] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:24.729063 30900 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 07:33:24.729461 1936 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:24.730490 1994 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.731289 2155 log.cc:924] Time spent T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.051s user 0.001s sys 0.001s
I20260809 07:33:24.732715 1994 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.722493 2032 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.733243 1663 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
W20260809 07:33:24.734231 2132 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.092s user 0.001s sys 0.000s
I20260809 07:33:24.737358 31177 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:24.741658 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:24.743279 31177 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.747920 30891 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:24.748210 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.749001 31054 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:24.750061 31005 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.716075 2131 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:24.766124 31054 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:24.696417 2009 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.752061 2078 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.771420 2131 consensus_queue.cc:1046] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:24.773317 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:24.767274 30852 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:24.712913 1935 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260809 07:33:24.787302 31045 pending_rounds.cc:76] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:24.717032 2135 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.752507 1566 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260809 07:33:24.757248 1827 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:24.792984 31045 pending_rounds.cc:97] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260809 07:33:24.801889 31045 raft_consensus.cc:2866] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:24.734779 1521 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.805934 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.813369 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:24.814030 31058 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:24.805647 1544 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:24.746961 2061 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.786413 31177 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:24.810914 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:24.751716 1994 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260809 07:33:24.825569 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:24.822199 31165 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260809 07:33:24.824556 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:24.826331 31165 pending_rounds.cc:76] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:24.827760 31165 pending_rounds.cc:97] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260809 07:33:24.828003 31165 raft_consensus.cc:2866] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:24.821381 30904 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:24.796939 2023 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:24.797122 2099 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:24.833211 31005 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.838016 31182 raft_consensus.cc:1226] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:24.815732 2044 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:24.816073 2006 raft_consensus.cc:3032] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
W20260809 07:33:24.897889 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 64756.8us
I20260809 07:33:24.844156 2131 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:24.827080 2135 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:24.800810 2067 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260809 07:33:24.819576 2032 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 07:33:24.899381 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:24.802181 2098 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.898550 2006 raft_consensus.cc:729] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:24.842427 2044 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:24.938301 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:24.943416 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:24.944149 31177 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:24.946599 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:24.839411 2099 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:24.914950 2097 log.cc:924] Time spent T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.072s user 0.000s sys 0.005s
I20260809 07:33:24.846033 1621 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260809 07:33:24.958339 31169 raft_consensus.cc:1226] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
W20260809 07:33:24.958547 2105 log.cc:924] Time spent T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.660s user 0.003s sys 0.000s
I20260809 07:33:24.938926 2008 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:24.939987 2044 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:25.021867 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 123885us
I20260809 07:33:24.927979 1529 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260809 07:33:24.900620 2078 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:24.956568 2032 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:24.940726 1820 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260809 07:33:25.156327 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 84029.1us
I20260809 07:33:24.964325 2177 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:25.171866 30896 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s user 0.012s sys 0.037s
I20260809 07:33:25.070652 2094 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:25.183998 30896 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.534s user 0.013s sys 0.037s
I20260809 07:33:25.184337 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:24.910650 1544 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.021071 2006 consensus_queue.cc:260] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.171893 1521 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:25.151480 1926 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260809 07:33:25.187834 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1324) took 1169 ms. Trace:
I20260809 07:33:25.190802 31046 rpcz_store.cc:276] 0809 07:33:24.018283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.018622 (+ 339us) service_pool.cc:227] Handling call
0809 07:33:24.018947 (+ 325us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:24.038956 (+ 20009us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:24.038988 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:24.039422 (+ 434us) log.cc:842] Serialized 26 byte log entry
0809 07:33:24.228935 (+189513us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:24.228970 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:24.229250 (+ 280us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:24.779835 (+550585us) spinlock_profiling.cc:243] Waited 9.11 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:25.181976 (+402141us) raft_consensus.cc:1671] finished
0809 07:33:25.181999 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:25.187758 (+ 5759us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19569,"raft.queue_time_us":20729,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"spinlock_wait_cycles":20953472,"thread_start_us":189185,"threads_started":1,"wal-append.queue_time_us":255610}
I20260809 07:33:25.181348 2032 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:25.219172 2154 log.cc:924] Time spent T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.569s user 0.000s sys 0.004s
I20260809 07:33:25.199188 2032 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:25.190055 2061 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.198455 2191 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:25.210196 1822 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260809 07:33:25.263561 2152 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.612s user 0.002s sys 0.000s
W20260809 07:33:25.264689 2176 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.190s user 0.000s sys 0.001s
W20260809 07:33:25.265328 1936 consensus_meta.cc:287] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.535s user 0.005s sys 0.004s
I20260809 07:33:25.190418 2177 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.240006 1521 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.266911 1521 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.267803 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:25.252501 2098 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:25.254441 2165 log.cc:924] Time spent T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.233s user 0.000s sys 0.001s
W20260809 07:33:25.238694 2164 log.cc:924] Time spent T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.338s user 0.001s sys 0.000s
W20260809 07:33:25.256073 1663 consensus_meta.cc:287] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.522s user 0.011s sys 0.000s
W20260809 07:33:25.256842 2188 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.055s user 0.000s sys 0.002s
I20260809 07:33:25.205156 2149 consensus_queue.cc:1046] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260809 07:33:25.288364 31170 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:25.288856 31164 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: []
W20260809 07:33:25.259719 2181 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.081s user 0.001s sys 0.001s
I20260809 07:33:25.260258 31182 raft_consensus.cc:1261] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260809 07:33:25.291756 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 50373.3us
W20260809 07:33:25.263331 2157 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.306s user 0.001s sys 0.001s
I20260809 07:33:25.235297 2099 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:25.268430 2177 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:25.240429 2031 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.294590 31177 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:25.270255 1521 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:25.309705 2172 log.cc:924] Time spent T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.157s user 0.001s sys 0.000s
W20260809 07:33:25.313630 2185 log.cc:924] Time spent T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.078s user 0.000s sys 0.001s
W20260809 07:33:25.315896 2169 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.127s user 0.000s sys 0.000s
I20260809 07:33:25.317173 2031 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:25.318540 2178 log.cc:924] Time spent T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.070s user 0.000s sys 0.001s
I20260809 07:33:25.319290 2031 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.308616 2198 consensus_queue.cc:1046] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
W20260809 07:33:25.321861 2174 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.114s user 0.001s sys 0.000s
W20260809 07:33:25.323956 2173 log.cc:924] Time spent T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260809 07:33:25.292006 1898 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:25.325071 1898 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:25.325404 1898 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:25.325456 2149 raft_consensus.cc:3032] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:25.329376 2149 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:25.330704 2149 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.336292 2131 consensus_queue.cc:1046] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:25.336787 2149 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:25.338083 2196 consensus_queue.cc:1046] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260809 07:33:25.339279 2096 raft_consensus.cc:3032] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:25.340073 2196 consensus_queue.cc:1225] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260809 07:33:25.340910 2156 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.429s user 0.003s sys 0.000s
I20260809 07:33:25.342626 2131 raft_consensus.cc:3032] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:25.343263 2131 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:25.342447 2006 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:25.349426 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1153) took 1180 ms. Trace:
I20260809 07:33:25.349787 31174 rpcz_store.cc:276] 0809 07:33:24.169136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.169257 (+ 121us) service_pool.cc:227] Handling call
0809 07:33:24.169533 (+ 276us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:24.621352 (+451819us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.349175 (+727823us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54003453d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:25.349366 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1004164480}
I20260809 07:33:25.347616 1521 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.347535 2031 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.357335 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1158) took 1185 ms. Trace:
I20260809 07:33:25.357717 31165 rpcz_store.cc:276] 0809 07:33:24.172030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.217802 (+ 45772us) service_pool.cc:227] Handling call
0809 07:33:24.821859 (+604057us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:24.828392 (+ 6533us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.352621 (+524229us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b540026a2d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:25.357275 (+ 4654us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1388561152}
I20260809 07:33:25.366124 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:25.367710 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:25.339617 2200 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260809 07:33:25.366024 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 419489us
I20260809 07:33:25.372956 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:25.374411 31177 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:25.348579 2096 raft_consensus.cc:729] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:25.353446 2031 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.387792 2096 consensus_queue.cc:260] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.353416 1521 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.390739 2177 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:25.379281 31054 raft_consensus.cc:3032] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:25.394483 31054 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:25.380539 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:25.365628 2009 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s user 0.005s sys 0.015s
I20260809 07:33:25.385370 2078 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:25.386615 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.397564 30896 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:25.384397 2061 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:25.407420 30911 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:25.415897 30904 raft_consensus.cc:1261] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:25.417572 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:25.369626 1994 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.395381 31054 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.434768 31054 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:25.435583 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:25.409828 2009 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s user 0.005s sys 0.015s
I20260809 07:33:25.390065 2031 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.390604 2196 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260809 07:33:25.436225 31048 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:25.400593 2019 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.399987 2067 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.489255 31054 raft_consensus.cc:1261] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 07:33:25.378304 1195 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.448431 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.491750 31045 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:25.451718 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:25.422345 2191 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.367329 2092 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260809 07:33:25.443619 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:25.502646 31181 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:25.421939 2200 consensus_queue.cc:1225] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:25.420930 1800 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.421211 2061 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:25.426362 2117 raft_consensus.cc:3032] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:25.427706 1949 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.433027 1856 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.520398 2117 raft_consensus.cc:729] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:25.444628 30890 raft_consensus.cc:1226] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260809 07:33:25.503741 2196 consensus_queue.cc:1225] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260809 07:33:25.497645 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:25.498202 2019 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.504324 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.354990 2131 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.445463 1521 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:25.517663 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.519706 2061 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:25.493166 31054 pending_rounds.cc:76] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 2
I20260809 07:33:25.572535 31054 pending_rounds.cc:97] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 07:33:25.572880 31054 raft_consensus.cc:2866] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:25.532819 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:25.510982 1926 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.446271 1994 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.518800 2200 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.580267 31045 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:25.506776 2067 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.504276 1195 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.536355 2008 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.602029 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:25.600520 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 181863us
I20260809 07:33:25.581575 2023 raft_consensus.cc:686] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:25.623687 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:25.511509 2191 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.519527 1800 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.645305 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:25.607348 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 91493.1us
W20260809 07:33:25.600414 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 84322.7us
I20260809 07:33:25.540362 1856 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.640627 31174 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:25.631511 31006 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:25.642093 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:25.631060 2196 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.540128 1949 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.651489 31174 raft_consensus.cc:595] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
W20260809 07:33:25.639102 2135 consensus_meta.cc:287] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.726s user 0.009s sys 0.003s
I20260809 07:33:25.612449 2019 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.658803 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:25.645202 2078 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:25.592343 1926 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.639715 2191 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:25.638914 1994 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.668375 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:25.671542 2044 consensus_meta.cc:287] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.622s user 0.006s sys 0.006s
I20260809 07:33:25.684113 1195 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.667667 30899 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:25.696732 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1344) took 1104 ms. Trace:
I20260809 07:33:25.664664 1901 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:25.712158 31051 rpcz_store.cc:276] 0809 07:33:24.589497 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.589624 (+ 127us) service_pool.cc:227] Handling call
0809 07:33:24.600241 (+ 10617us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:24.644200 (+ 43959us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:24.644230 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.257021 (+612791us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b2c00000238. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:33:25.257420 (+ 399us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.354027 (+ 96607us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:25.354049 (+ 22us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.354102 (+ 53us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:25.693146 (+339044us) raft_consensus.cc:1671] finished
0809 07:33:25.693168 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:25.693391 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":55356,"raft.queue_time_us":50739,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7906,"spinlock_wait_cycles":5438720,"thread_start_us":96280,"threads_started":1,"wal-append.queue_time_us":179623}
I20260809 07:33:25.595384 2200 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.703462 30890 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:25.632086 2008 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.687598 2098 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:25.664722 2234 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:25.717236 31058 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:25.640961 2023 consensus_queue.cc:237] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.692217 2078 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:25.664611 1822 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
I20260809 07:33:25.713790 1994 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.777382 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:25.712359 2205 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.201s user 0.000s sys 0.002s
I20260809 07:33:25.700990 1544 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.668056 1898 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.775278 31174 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:25.720732 1901 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:25.788724 2217 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.789223 2217 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.602171 2117 consensus_queue.cc:260] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.780429 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:25.782241 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 123034us
I20260809 07:33:25.730633 2013 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:25.657886 2196 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.786778 31174 raft_consensus.cc:595] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:25.786973 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:25.756032 2234 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:25.744877 1820 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.774328 2078 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:25.793771 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.777851 1517 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:25.720860 2044 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.764025 1800 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.805119 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.810863 1856 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:25.811398 2211 log.cc:924] Time spent T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.229s user 0.000s sys 0.001s
I20260809 07:33:25.812516 1949 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.813073 2067 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.814064 2235 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.785247 1994 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.815667 2196 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:25.790302 2228 consensus_queue.cc:1035] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 07:33:25.792613 1901 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:25.855324 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 61601.9us
I20260809 07:33:25.857806 31174 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e327743625143d2b7f941cececc7d75"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:25.817751 2227 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:25.859050 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:25.862033 31174 raft_consensus.cc:595] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:25.861568 31045 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:25.862432 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:25.862509 31174 raft_consensus.cc:603] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:25.861143 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.868010 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:25.801729 2234 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:25.848519 1663 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:25.848938 1517 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.019387 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:25.862249 30890 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:25.870798 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:26.067551 30896 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:25.818440 2135 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:25.793211 1544 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.859512 1949 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:26.043058 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:26.028179 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:26.045550 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 261098us
I20260809 07:33:26.046243 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:26.029286 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:26.049757 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:26.028009 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:25.914707 31051 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:25.797497 2013 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.804672 1820 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:26.045281 2032 consensus_meta.cc:287] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.810s user 0.008s sys 0.004s
I20260809 07:33:26.079771 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:26.174847 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:26.175653 30890 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:25.896701 2067 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.182s
I20260809 07:33:26.053418 1856 consensus_queue.cc:1035] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260809 07:33:26.183394 31054 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:26.183774 1517 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.188283 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:26.177721 2188 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.115s user 0.000s sys 0.001s
I20260809 07:33:26.078622 1926 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.070381 2217 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.050369 2061 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:26.226187 2218 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.245s user 0.000s sys 0.002s
W20260809 07:33:26.230444 2210 log.cc:924] Time spent T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.474s user 0.005s sys 0.005s
I20260809 07:33:26.174589 2032 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:25.859648 2235 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.293119 30890 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:26.466739 30890 raft_consensus.cc:595] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:26.079114 2135 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.223896 2214 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:26.458802 31051 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:26.472643 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1355) took 1033 ms. Trace:
I20260809 07:33:26.473042 31051 rpcz_store.cc:276] 0809 07:33:25.439236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.205180 (+765944us) service_pool.cc:227] Handling call
0809 07:33:26.458439 (+253259us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.472348 (+ 13909us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.472576 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.234638 2223 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.148699 1949 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.064503 2227 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.488740 31169 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:26.488870 2031 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Stepping down as leader of term 9
I20260809 07:33:26.489265 2031 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:26.489853 2227 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.494092 30900 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:26.413147 2008 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:26.479408 2149 consensus_meta.cc:287] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 1.141s user 0.013s sys 0.005s
I20260809 07:33:26.466998 2217 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:26.495679 2246 log.cc:924] Time spent T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.423s user 0.000s sys 0.000s
I20260809 07:33:26.471150 1898 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:26.561318 2254 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
W20260809 07:33:26.565708 2263 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.114s user 0.000s sys 0.000s
I20260809 07:33:26.495980 2044 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.578639 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1013) took 1177 ms. Trace:
I20260809 07:33:26.569188 31182 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
W20260809 07:33:26.570696 2248 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.379s user 0.000s sys 0.001s
I20260809 07:33:26.572692 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1218) took 1143 ms. Trace:
I20260809 07:33:26.573770 1750 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:26.574879 2237 log.cc:924] Time spent T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.452s user 0.001s sys 0.000s
I20260809 07:33:26.575284 2094 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.490070 2031 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.494797 2251 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:26.578226 2044 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:26.579067 30891 rpcz_store.cc:276] 0809 07:33:25.401080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.402705 (+ 1625us) service_pool.cc:227] Handling call
0809 07:33:25.403041 (+ 336us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.411973 (+ 8932us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.412001 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.428018 (+ 16017us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.800301 (+372283us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:25.800612 (+ 311us) log.cc:842] Serialized 14 byte log entry
0809 07:33:25.800768 (+ 156us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.800836 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:26.578387 (+777551us) raft_consensus.cc:1671] finished
0809 07:33:26.578406 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.578550 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28335,"prepare.queue_time_us":242601,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":409990,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":111388,"thread_start_us":316656,"threads_started":1,"wal-append.queue_time_us":720169}
I20260809 07:33:26.582513 30911 rpcz_store.cc:276] 0809 07:33:25.429482 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.431072 (+ 1590us) service_pool.cc:227] Handling call
0809 07:33:25.431328 (+ 256us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:25.667252 (+235924us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:25.685229 (+ 17977us) log.cc:842] Serialized 14 byte log entry
0809 07:33:25.766281 (+ 81052us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:25.766364 (+ 83us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:25.766607 (+ 243us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.572439 (+805832us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54001b4ed4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:26.572487 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.572624 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10408,"prepare.queue_time_us":17786,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":59442,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":67,"spinlock_wait_cycles":542059904,"thread_start_us":68772,"threads_started":1,"wal-append.queue_time_us":338805}
I20260809 07:33:26.605888 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1225) took 1164 ms. Trace:
I20260809 07:33:26.606230 30911 rpcz_store.cc:276] 0809 07:33:25.441684 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.605084 (+1163400us) service_pool.cc:227] Handling call
0809 07:33:26.605393 (+ 309us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.605566 (+ 173us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.605591 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.605631 (+ 40us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:26.605666 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.605715 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.605831 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:26.574936 1827 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260809 07:33:26.608899 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1175) took 1237 ms. Trace:
I20260809 07:33:26.598407 1750 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.601155 2094 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:26.615636 2255 consensus_queue.cc:1159] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c9de56c3fa3487c9dac990675c34be5" responder_term: 10 status { last_received { term: 8 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260809 07:33:26.608525 2251 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.612795 2031 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:26.619634 2251 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.621793 1949 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.493335 2214 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.613039 2044 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.623060 2061 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.624380 2044 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.609308 31165 rpcz_store.cc:276] 0809 07:33:25.371643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.375673 (+ 4030us) service_pool.cc:227] Handling call
0809 07:33:25.376024 (+ 351us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.381988 (+ 5964us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.382030 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.382519 (+ 489us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.432285 (+ 49766us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:25.433230 (+ 945us) log.cc:842] Serialized 14 byte log entry
0809 07:33:25.504203 (+ 70973us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.504278 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:26.608690 (+1104412us) raft_consensus.cc:1671] finished
0809 07:33:26.608713 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.608829 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":76229,"prepare.queue_time_us":675,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":86161,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":104,"spinlock_wait_cycles":7435648,"thread_start_us":49448,"threads_started":1,"wal-append.queue_time_us":100462}
I20260809 07:33:26.618036 2094 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.559183 2200 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.591380 2196 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.661789 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1246) took 1282 ms. Trace:
I20260809 07:33:26.670236 31049 rpcz_store.cc:276] 0809 07:33:25.379065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.379189 (+ 124us) service_pool.cc:227] Handling call
0809 07:33:25.379491 (+ 302us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.448316 (+ 68825us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.448342 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.459621 (+ 11279us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.710189 (+250568us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:25.710231 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.710306 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:26.625396 (+915090us) raft_consensus.cc:1671] finished
0809 07:33:26.625419 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.661705 (+ 36286us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34976,"raft.queue_time_us":41880,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":12822,"spinlock_wait_cycles":1611648,"thread_start_us":285648,"threads_started":2,"wal-append.queue_time_us":360418}
I20260809 07:33:26.673157 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1357) took 1205 ms. Trace:
I20260809 07:33:26.673573 31049 rpcz_store.cc:276] 0809 07:33:25.468225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.672335 (+1204110us) service_pool.cc:227] Handling call
0809 07:33:26.672662 (+ 327us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.672857 (+ 195us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.672880 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.672919 (+ 39us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:26.672952 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.672999 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.673102 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.675302 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1358) took 1191 ms. Trace:
I20260809 07:33:26.678401 31049 rpcz_store.cc:276] 0809 07:33:25.483936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.674568 (+1190632us) service_pool.cc:227] Handling call
0809 07:33:26.674863 (+ 295us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.675030 (+ 167us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.675054 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.675089 (+ 35us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:26.675120 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.675165 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.675255 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.680027 31049 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:26.680603 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1260) took 1055 ms. Trace:
I20260809 07:33:26.700099 31049 rpcz_store.cc:276] 0809 07:33:25.625257 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.679454 (+1054197us) service_pool.cc:227] Handling call
0809 07:33:26.679705 (+ 251us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.680392 (+ 687us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.680556 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.632227 30911 raft_consensus.cc:1261] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:26.701910 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1227) took 1259 ms. Trace:
I20260809 07:33:26.631314 1544 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.664427 2032 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:26.693622 2078 consensus_meta.cc:287] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.882s user 0.005s sys 0.007s
I20260809 07:33:26.695542 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1244) took 1319 ms. Trace:
W20260809 07:33:26.717725 30900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1021) took 1275 ms (client timeout 1617 ms). Trace:
I20260809 07:33:26.717051 31059 rpcz_store.cc:276] 0809 07:33:25.376519 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.376677 (+ 158us) service_pool.cc:227] Handling call
0809 07:33:25.377021 (+ 344us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.714735 (+337714us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.714767 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.744957 (+ 30190us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.808568 (+ 63611us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:25.808607 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.808692 (+ 85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:26.695335 (+886643us) raft_consensus.cc:1671] finished
0809 07:33:26.695355 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.695481 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":62376,"raft.queue_time_us":82189,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":27110,"thread_start_us":338078,"threads_started":2,"wal-append.queue_time_us":302891}
W20260809 07:33:26.663712 1901 consensus_meta.cc:287] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.640s user 0.006s sys 0.002s
I20260809 07:33:26.693745 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1020) took 1277 ms. Trace:
I20260809 07:33:26.702301 30911 rpcz_store.cc:276] 0809 07:33:25.442146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.607142 (+1164996us) service_pool.cc:227] Handling call
0809 07:33:26.631892 (+ 24750us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.701638 (+ 69746us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.701854 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.723457 30899 rpcz_store.cc:276] 0809 07:33:25.416683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.417125 (+ 442us) service_pool.cc:227] Handling call
0809 07:33:26.693571 (+1276446us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54001937d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:26.693681 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":562002816}
I20260809 07:33:26.590579 2217 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.551844 2272 consensus_queue.cc:1035] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.097s
I20260809 07:33:26.625172 2235 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.646782 2214 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:26.718189 30900 rpcz_store.cc:269] 0809 07:33:25.443707 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.451313 (+ 7606us) service_pool.cc:227] Handling call
0809 07:33:26.717476 (+1266163us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54003c28d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:26.717653 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2296634752}
I20260809 07:33:26.719666 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1262) took 1094 ms. Trace:
I20260809 07:33:26.756103 31059 rpcz_store.cc:276] 0809 07:33:25.625616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.718639 (+1093023us) service_pool.cc:227] Handling call
0809 07:33:26.718987 (+ 348us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.719307 (+ 320us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:26.719352 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.719407 (+ 55us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:26.719441 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.719497 (+ 56us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.719610 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3414,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139}
I20260809 07:33:26.496408 2223 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:26.729257 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1232) took 1214 ms. Trace:
I20260809 07:33:26.744264 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1233) took 1223 ms. Trace:
I20260809 07:33:26.758546 30899 rpcz_store.cc:276] 0809 07:33:25.520842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.733955 (+1213113us) service_pool.cc:227] Handling call
0809 07:33:26.743750 (+ 9795us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.743936 (+ 186us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.743960 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.744012 (+ 52us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:26.744047 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.744094 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.744200 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:26.719643 2240 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.256s user 0.006s sys 0.000s
I20260809 07:33:26.746593 1901 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:26.750046 31004 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.899s user 0.042s sys 0.081s
I20260809 07:33:26.758430 30911 rpcz_store.cc:276] 0809 07:33:25.514658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.728329 (+1213671us) service_pool.cc:227] Handling call
0809 07:33:26.728701 (+ 372us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.728894 (+ 193us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.728919 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.728973 (+ 54us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:26.729010 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.729067 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.729187 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:26.765172 31004 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.914s user 0.042s sys 0.081s
I20260809 07:33:26.703085 1820 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.791364 30911 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:26.792176 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1237) took 1196 ms. Trace:
I20260809 07:33:26.792502 30911 rpcz_store.cc:276] 0809 07:33:25.595934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.765400 (+1169466us) service_pool.cc:227] Handling call
0809 07:33:26.790975 (+ 25575us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.791879 (+ 904us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.792110 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.661803 1750 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.805473 30900 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:26.614817 2013 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.831575 30899 raft_consensus.cc:1261] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 07:33:26.832443 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1235) took 1269 ms. Trace:
I20260809 07:33:26.832845 30899 rpcz_store.cc:276] 0809 07:33:25.562975 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.759797 (+1196822us) service_pool.cc:227] Handling call
0809 07:33:26.831223 (+ 71426us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.832160 (+ 937us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.832382 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.812229 2230 consensus_queue.cc:1046] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.248s
I20260809 07:33:26.755286 2244 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:26.806120 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1234) took 1264 ms. Trace:
I20260809 07:33:26.835783 30900 rpcz_store.cc:276] 0809 07:33:25.541681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.755768 (+1214087us) service_pool.cc:227] Handling call
0809 07:33:26.805128 (+ 49360us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.805859 (+ 731us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.806058 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.834653 2149 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:26.840397 1621 raft_consensus.cc:979] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: : Instructing follower a720817fc113446a8ed55e61523b280a to start an election
I20260809 07:33:26.841284 31175 raft_consensus.cc:2379] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:26.840560 2149 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.841537 1901 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.844775 2131 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:26.805305 2032 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:26.734462 2078 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.830754 2098 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.914600 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:43208 (request call id 64) took 1478 ms. Trace:
I20260809 07:33:26.915153 30899 rpcz_store.cc:276] 0809 07:33:25.440469 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.833687 (+1393218us) service_pool.cc:227] Handling call
0809 07:33:26.910384 (+ 76697us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:26.909008 2092 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.917886 2092 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:26.918890 2236 log.cc:924] Time spent T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 1.030s user 0.008s sys 0.000s
I20260809 07:33:26.914459 2094 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:26.900063 1901 raft_consensus.cc:1067] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Signalling peer a720817fc113446a8ed55e61523b280a to start an election
W20260809 07:33:26.903213 2240 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.097s user 0.000s sys 0.001s
I20260809 07:33:26.848743 2149 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.714594 2196 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.917703 2276 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.924165 2094 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.922672 2098 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:26.950709 2224 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.087s user 0.000s sys 0.004s
I20260809 07:33:26.951287 2094 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.926298 2078 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:26.912120 2198 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:26.952474 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1178) took 1579 ms. Trace:
I20260809 07:33:26.952840 31172 rpcz_store.cc:276] 0809 07:33:25.372538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.375310 (+ 2772us) service_pool.cc:227] Handling call
0809 07:33:25.410532 (+ 35222us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.438126 (+ 27594us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.438159 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.438576 (+ 417us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.519231 (+ 80655us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:25.519270 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.519337 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:26.732378 (+1213041us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:26.952135 (+219757us) raft_consensus.cc:1671] finished
0809 07:33:26.952159 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.952402 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":64398,"raft.queue_time_us":31816,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11602,"spinlock_wait_cycles":9017216,"thread_start_us":80281,"threads_started":1,"wal-append.queue_time_us":137768}
I20260809 07:33:26.947072 2092 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.945243 2276 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.957686 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1353) took 1529 ms. Trace:
I20260809 07:33:26.959959 31054 rpcz_store.cc:276] 0809 07:33:25.428192 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.197966 (+769774us) service_pool.cc:227] Handling call
0809 07:33:26.367917 (+169951us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.836095 (+468178us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.836128 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:26.836169 (+ 41us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:26.836202 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:26.957453 (+121251us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b540010b154. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:26.957503 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:26.957627 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1076218240}
I20260809 07:33:26.952876 2198 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.965819 2198 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.917809 2032 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:26.972035 2032 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:26.986842 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:26.987686 31054 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:27.002048 2261 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:27.001256 2033 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.011142 2033 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.017349 2098 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:27.019793 31046 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:27.021297 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:27.006808 2247 log.cc:924] Time spent T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.268s user 0.000s sys 0.001s
I20260809 07:33:26.981809 2276 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.008068 2261 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:27.110071 31058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1240) took 2205 ms (client timeout 1646 ms). Trace:
W20260809 07:33:27.139562 31058 rpcz_store.cc:269] 0809 07:33:24.825188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.825334 (+ 146us) service_pool.cc:227] Handling call
0809 07:33:27.030232 (+2204898us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b5400012cd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:27.030374 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1980906752}
I20260809 07:33:27.140856 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1214) took 2333 ms. Trace:
I20260809 07:33:27.137485 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:27.143771 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1176) took 1513 ms. Trace:
I20260809 07:33:27.152783 31177 rpcz_store.cc:276] 0809 07:33:25.630577 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.630733 (+ 156us) service_pool.cc:227] Handling call
0809 07:33:25.637118 (+ 6385us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.648954 (+ 11836us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.648987 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.657433 (+ 8446us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.753938 (+ 96505us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:25.753978 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.754049 (+ 71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:27.143529 (+1389480us) raft_consensus.cc:1671] finished
0809 07:33:27.143555 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.143702 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3479,"raft.queue_time_us":3614,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":8021,"thread_start_us":55996,"threads_started":1,"wal-append.queue_time_us":582295}
I20260809 07:33:27.154982 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:27.155723 31058 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:27.157068 31054 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:27.145658 30906 rpcz_store.cc:276] 0809 07:33:24.807539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.807677 (+ 138us) service_pool.cc:227] Handling call
0809 07:33:24.833164 (+ 25487us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.417012 (+583848us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:25.417399 (+ 387us) log.cc:842] Serialized 14 byte log entry
0809 07:33:25.720655 (+303256us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.751538 (+ 30883us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.751674 (+ 136us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:25.751706 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.751773 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:27.140610 (+1388837us) raft_consensus.cc:1671] finished
0809 07:33:27.140634 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.140789 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":523742,"prepare.queue_time_us":298,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":638165,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":161,"thread_start_us":362527,"threads_started":2,"wal-append.queue_time_us":441692}
W20260809 07:33:27.159358 30898 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.087s user 0.009s sys 0.034s
W20260809 07:33:27.159691 30898 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.087s user 0.009s sys 0.034s
I20260809 07:33:27.160553 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1012) took 1762 ms. Trace:
I20260809 07:33:27.162918 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1264) took 1499 ms. Trace:
I20260809 07:33:27.163345 31058 rpcz_store.cc:276] 0809 07:33:25.663674 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.162066 (+1498392us) service_pool.cc:227] Handling call
0809 07:33:27.162444 (+ 378us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.162623 (+ 179us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:27.162647 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:27.162687 (+ 40us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:27.162720 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.162768 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.162860 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:27.165196 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1360) took 1460 ms. Trace:
I20260809 07:33:27.165520 31058 rpcz_store.cc:276] 0809 07:33:25.704575 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.164417 (+1459842us) service_pool.cc:227] Handling call
0809 07:33:27.164731 (+ 314us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.164906 (+ 175us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:27.164929 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:27.164981 (+ 52us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:27.165012 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.165057 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.165147 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:27.169140 30909 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.126s user 0.015s sys 0.028s
W20260809 07:33:27.171811 30909 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.128s user 0.015s sys 0.028s
I20260809 07:33:27.188118 31058 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260809 07:33:27.188710 31058 pending_rounds.cc:76] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 6
I20260809 07:33:27.188992 31058 pending_rounds.cc:97] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260809 07:33:27.189309 31058 raft_consensus.cc:2866] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:27.190032 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1266) took 1458 ms. Trace:
I20260809 07:33:27.190421 31058 rpcz_store.cc:276] 0809 07:33:25.732702 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.187354 (+1454652us) service_pool.cc:227] Handling call
0809 07:33:27.187726 (+ 372us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.189776 (+ 2050us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.189982 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:27.192021 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1267) took 1458 ms. Trace:
I20260809 07:33:27.192474 31058 rpcz_store.cc:276] 0809 07:33:25.733147 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.191226 (+1458079us) service_pool.cc:227] Handling call
0809 07:33:27.191518 (+ 292us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.191686 (+ 168us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:27.191710 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:27.191750 (+ 40us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:27.191787 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.191833 (+ 46us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.191928 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:27.233191 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1359) took 1534 ms. Trace:
I20260809 07:33:27.233711 31054 rpcz_store.cc:276] 0809 07:33:25.698727 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.161987 (+1463260us) service_pool.cc:227] Handling call
0809 07:33:27.232627 (+ 70640us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.232844 (+ 217us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:27.232870 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:27.232911 (+ 41us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:27.232948 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.232998 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.233120 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:27.160974 30898 rpcz_store.cc:276] 0809 07:33:25.398222 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.398739 (+ 517us) service_pool.cc:227] Handling call
0809 07:33:25.399082 (+ 343us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.399324 (+ 242us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.399350 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.399762 (+ 412us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.855238 (+455476us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:27.160026 (+1304788us) log.cc:842] Serialized 14 byte log entry
0809 07:33:27.160254 (+ 228us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.160327 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:27.160370 (+ 43us) raft_consensus.cc:1671] finished
0809 07:33:27.160380 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.160498 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":187798,"prepare.queue_time_us":344016,"prepare.run_cpu_time_us":3744,"prepare.run_wall_time_us":173724,"raft.queue_time_us":150,"thread_start_us":1498430,"threads_started":2,"wal-append.queue_time_us":656475}
I20260809 07:33:27.243562 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1183) took 1372 ms. Trace:
I20260809 07:33:27.243382 2253 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.555301 31182 rpcz_store.cc:276] 0809 07:33:25.871479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.871948 (+ 469us) service_pool.cc:227] Handling call
0809 07:33:27.243357 (+1371409us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54003a07d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:27.243492 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1123491584}
I20260809 07:33:27.311764 2196 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.390067 2191 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:27.527328 2096 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:27.313005 2223 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
W20260809 07:33:27.501009 2265 log.cc:924] Time spent T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.442s user 0.000s sys 0.005s
I20260809 07:33:27.301653 1994 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:27.509513 2234 consensus_meta.cc:287] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 1.348s user 0.010s sys 0.002s
W20260809 07:33:27.427260 2266 log.cc:924] Time spent T 00000000000000000000000000000000 P 8bb995981efc41a7a744904b9759529a: Append to log took a long time: real 0.762s user 0.000s sys 0.008s
W20260809 07:33:27.245083 30848 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.081s user 0.025s sys 0.038s
I20260809 07:33:27.298306 2250 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:27.564347 30848 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.502s user 0.037s sys 0.042s
W20260809 07:33:27.430094 30849 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.336s user 0.021s sys 0.031s
W20260809 07:33:27.574599 30849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.481s user 0.021s sys 0.032s
I20260809 07:33:27.311741 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1256) took 1909 ms. Trace:
I20260809 07:33:27.520713 2117 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:27.559422 2234 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:27.568570 31046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1249) took 2188 ms (client timeout 1876 ms). Trace:
W20260809 07:33:27.583230 31046 rpcz_store.cc:269] 0809 07:33:25.380023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.380141 (+ 118us) service_pool.cc:227] Handling call
0809 07:33:27.568276 (+2188135us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400108e54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:27.568483 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3717783936}
I20260809 07:33:27.575973 31045 rpcz_store.cc:276] 0809 07:33:25.402004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.585159 (+183155us) service_pool.cc:227] Handling call
0809 07:33:25.622073 (+ 36914us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:25.862707 (+240634us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.311457 (+1448750us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54000dcad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:27.311659 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":550305024}
I20260809 07:33:27.567840 2253 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.528231 2135 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.587229 2061 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:27.343217 2216 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.811s user 0.024s sys 0.038s
I20260809 07:33:27.589002 1949 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.530499 2278 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.521266 2255 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:27.582939 2263 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.394s user 0.004s sys 0.009s
I20260809 07:33:27.592465 1949 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.593231 2258 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:27.588792 2061 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.428777 1898 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:27.249508 2217 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.527179 2033 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.243157 2261 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:27.628788 31175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1187) took 1750 ms (client timeout 1528 ms). Trace:
W20260809 07:33:27.591231 2216 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.059s user 0.028s sys 0.041s
I20260809 07:33:27.582654 1994 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:27.659600 2243 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.629122 2228 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.660915 31182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b4960950e96549f499238d58397046ce"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
W20260809 07:33:27.674175 30904 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.601s user 0.009s sys 0.084s
I20260809 07:33:27.557559 2191 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:27.685065 31175 rpcz_store.cc:269] 0809 07:33:25.878795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.049489 (+170694us) service_pool.cc:227] Handling call
0809 07:33:27.628498 (+1579009us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54003f0a54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:27.628672 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1742633344}
W20260809 07:33:27.690752 30904 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.618s user 0.009s sys 0.084s
I20260809 07:33:27.693933 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1221) took 2263 ms. Trace:
W20260809 07:33:27.684407 1855 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.053s user 0.016s sys 0.074s
I20260809 07:33:27.706508 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1251) took 2310 ms. Trace:
I20260809 07:33:27.786842 31057 rpcz_store.cc:276] 0809 07:33:25.395967 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.396611 (+ 644us) service_pool.cc:227] Handling call
0809 07:33:25.396953 (+ 342us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.425662 (+ 28709us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.425687 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.426087 (+ 400us) log.cc:842] Serialized 26 byte log entry
0809 07:33:25.855489 (+429402us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:25.855618 (+ 129us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:25.855813 (+ 195us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:27.186400 (+1330587us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b48000d1058. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:27.693676 (+507276us) raft_consensus.cc:1671] finished
0809 07:33:27.693694 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.706417 (+ 12723us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28353,"raft.queue_time_us":35525,"raft.run_cpu_time_us":8902,"raft.run_wall_time_us":1792532,"spinlock_wait_cycles":13004672,"thread_start_us":368745,"threads_started":1,"wal-append.queue_time_us":1265716}
I20260809 07:33:27.791363 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1220) took 2361 ms. Trace:
I20260809 07:33:27.685134 2217 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:27.794685 2233 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.720s user 0.036s sys 0.067s
W20260809 07:33:27.795117 2233 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.729s user 0.036s sys 0.067s
I20260809 07:33:27.694355 30904 rpcz_store.cc:276] 0809 07:33:25.430274 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.434780 (+ 4506us) service_pool.cc:227] Handling call
0809 07:33:25.435050 (+ 270us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:25.435223 (+ 173us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:25.483815 (+ 48592us) log.cc:842] Serialized 14 byte log entry
0809 07:33:25.883464 (+399649us) log.cc:842] Serialized 14 byte log entry
0809 07:33:26.072565 (+189101us) log.cc:842] Serialized 14 byte log entry
0809 07:33:27.693216 (+1620651us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:27.693272 (+ 56us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:27.693339 (+ 67us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.693533 (+ 194us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.693870 (+ 337us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":398178,"prepare.queue_time_us":931510,"prepare.run_cpu_time_us":219,"prepare.run_wall_time_us":19625,"raft.queue_time_us":1289263,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":18450,"thread_start_us":1936540,"threads_started":4,"wal-append.queue_time_us":239259,"wal-append.run_cpu_time_us":2933,"wal-append.run_wall_time_us":1498844}
I20260809 07:33:27.689419 2261 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:27.706656 1855 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.086s user 0.016s sys 0.074s
I20260809 07:33:27.674387 2255 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.680338 1898 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.591467 2250 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.789122 1936 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:27.692768 1994 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.852512 31182 raft_consensus.cc:595] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:27.852933 31182 raft_consensus.cc:603] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:27.661685 2278 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.693663 2228 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.832299 2261 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.868121 2216 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:27.831768 2217 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:27.791814 30909 rpcz_store.cc:276] 0809 07:33:25.430009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.430110 (+ 101us) service_pool.cc:227] Handling call
0809 07:33:25.430423 (+ 313us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:25.560836 (+130413us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:25.561273 (+ 437us) log.cc:842] Serialized 14 byte log entry
0809 07:33:26.815751 (+1254478us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b240018b048. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091e29cf4 00007f4091e27dfb 00007f4091eb6559 00007f4091eb61e5 00007f4091eb582d 00007f4094b6446e 00007f4094b7bc0c 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551
0809 07:33:27.175164 (+359413us) log.cc:842] Serialized 14 byte log entry
0809 07:33:27.180583 (+ 5419us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:27.180640 (+ 57us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:27.182164 (+ 1524us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:27.791054 (+608890us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400052154. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:27.791115 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:27.791293 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":205068,"prepare.queue_time_us":340639,"prepare.run_cpu_time_us":3898,"prepare.run_wall_time_us":144424,"raft.queue_time_us":100652,"raft.run_cpu_time_us":570,"raft.run_wall_time_us":82572,"spinlock_wait_cycles":422232576,"thread_start_us":1424353,"threads_started":3,"wal-append.queue_time_us":222353}
I20260809 07:33:27.852665 1936 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.922583 2250 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.914444 2191 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:27.915665 2061 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.916392 1949 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.684337 2258 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.929410 2258 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.697103 2243 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.920323 2255 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.878206 2278 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.898970 2234 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.943562 2228 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.945386 1898 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:27.948916 2261 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.956663 2200 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:27.961374 2217 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:27.939124 2253 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:27.939477 1936 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.940802 1994 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.957371 2200 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.901145 2216 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:27.969707 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 07:33:27.971673 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:27.972414 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:27.974466 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:27.975476 2216 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.975617 32198 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:27.978989 2200 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:27.979919 2243 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:27.983899 2213 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.100s user 0.025s sys 0.043s
W20260809 07:33:27.984292 2213 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.316s user 0.025s sys 0.043s
W20260809 07:33:28.001634 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.210s user 0.003s sys 0.031s
W20260809 07:33:28.012358 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.221s user 0.003s sys 0.031s
I20260809 07:33:27.999307 2255 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:28.029201 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 2.24872e+06us
I20260809 07:33:28.064203 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:28.064919 30898 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:28.066030 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.066993 30900 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 4.
I20260809 07:33:28.069258 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.069888 30900 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:28.070686 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.071218 30900 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:28.071964 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.073001 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:28.073534 30900 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:28.074970 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.075776 30904 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:28.076754 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:28.077417 30904 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:28.080554 31049 raft_consensus.cc:1261] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260809 07:33:28.080938 31049 pending_rounds.cc:76] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 4
I20260809 07:33:28.081156 31049 pending_rounds.cc:97] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260809 07:33:28.081394 31049 raft_consensus.cc:2866] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:28.028919 2255 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.093639 30898 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:28.097455 30899 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:28.098179 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.108330 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1261) took 2482 ms. Trace:
I20260809 07:33:28.108793 31049 rpcz_store.cc:276] 0809 07:33:25.625443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.700957 (+1075514us) service_pool.cc:227] Handling call
0809 07:33:26.701267 (+ 310us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:28.081829 (+1380562us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.108083 (+ 26254us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540031c754. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:28.108263 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3171306752}
W20260809 07:33:28.097252 2023 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.511s user 0.036s sys 0.036s
W20260809 07:33:28.116539 31170 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.688s user 0.012s sys 0.016s
I20260809 07:33:28.079377 2253 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.082060 1958 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.082258 2150 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.117425 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.088227 1898 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.119468 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.096933 2191 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.123050 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:28.115820 2023 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.529s user 0.036s sys 0.036s
I20260809 07:33:28.141602 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:28.143882 31117 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.572s user 0.018s sys 0.041s
I20260809 07:33:28.146492 30899 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260809 07:33:28.147190 31117 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.710s user 0.018s sys 0.042s
I20260809 07:33:28.121757 1958 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.157071 1958 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:28.161604 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.740s user 0.002s sys 0.017s
I20260809 07:33:28.123035 2253 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:28.147746 2067 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.500s user 0.016s sys 0.048s
I20260809 07:33:28.123507 1898 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.121637 2150 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.124575 2191 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:28.116889 31170 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.689s user 0.012s sys 0.016s
I20260809 07:33:28.171903 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1185) took 2299 ms. Trace:
W20260809 07:33:28.166232 30890 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.683s user 0.008s sys 0.012s
W20260809 07:33:28.161955 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.740s user 0.002s sys 0.017s
W20260809 07:33:28.174268 30890 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.691s user 0.009s sys 0.012s
W20260809 07:33:28.159759 2215 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.362s user 0.018s sys 0.041s
W20260809 07:33:28.124454 31051 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.591s user 0.008s sys 0.020s
W20260809 07:33:28.169837 2067 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.520s user 0.016s sys 0.048s
W20260809 07:33:28.189144 31051 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.656s user 0.008s sys 0.020s
W20260809 07:33:28.162482 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.085s user 0.007s sys 0.015s
W20260809 07:33:28.190423 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.113s user 0.008s sys 0.015s
W20260809 07:33:28.172569 2229 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.475s user 0.015s sys 0.044s
I20260809 07:33:28.171797 1517 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:28.190025 2215 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.396s user 0.019s sys 0.041s
I20260809 07:33:28.200475 2023 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:28.200964 2253 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.190326 2250 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.203393 30904 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:28.172304 31170 rpcz_store.cc:276] 0809 07:33:25.872036 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.147432 (+275396us) service_pool.cc:227] Handling call
0809 07:33:26.184801 (+ 37369us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.185016 (+ 215us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:26.223462 (+ 38446us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.171526 (+1948064us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:28.171584 (+ 58us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:28.171639 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.171724 (+ 85us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.171844 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":237,"prepare.queue_time_us":38264,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":130309,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":16004,"thread_start_us":1743453,"threads_started":1,"wal-append.queue_time_us":64597,"wal-append.run_cpu_time_us":1015,"wal-append.run_wall_time_us":1084203}
I20260809 07:33:28.207430 2150 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.188064 1958 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:28.210940 30896 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.142s user 0.000s sys 0.027s
W20260809 07:33:28.198071 2229 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.502s user 0.015s sys 0.044s
W20260809 07:33:28.247783 30896 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.179s user 0.000s sys 0.027s
I20260809 07:33:28.115840 30900 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:28.236688 2117 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:28.252470 30891 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.627s user 0.017s sys 0.052s
W20260809 07:33:28.272866 30891 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.648s user 0.017s sys 0.053s
W20260809 07:33:28.274029 31048 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.498s user 0.000s sys 0.028s
W20260809 07:33:28.274458 31048 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.499s user 0.000s sys 0.028s
I20260809 07:33:28.274674 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1224) took 2833 ms. Trace:
I20260809 07:33:28.275439 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1254) took 2874 ms. Trace:
I20260809 07:33:28.275132 30891 rpcz_store.cc:276] 0809 07:33:25.441438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.604002 (+1162564us) service_pool.cc:227] Handling call
0809 07:33:26.604361 (+ 359us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.604563 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:26.625061 (+ 20498us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.274114 (+1649053us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:28.274170 (+ 56us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:28.274215 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.274288 (+ 73us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.274559 (+ 271us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":692,"prepare.queue_time_us":53462,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":1647958,"threads_started":1}
W20260809 07:33:28.277455 31055 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.638s user 0.000s sys 0.028s
W20260809 07:33:28.277777 31055 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.638s user 0.000s sys 0.028s
I20260809 07:33:28.213847 2253 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.213711 2255 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:28.291618 30911 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.431s user 0.006s sys 0.018s
W20260809 07:33:28.291960 30911 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.432s user 0.006s sys 0.018s
I20260809 07:33:28.211741 1517 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.238560 2200 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.251928 2150 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:28.289820 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.231s user 0.008s sys 0.020s
W20260809 07:33:28.327431 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.268s user 0.008s sys 0.020s
W20260809 07:33:28.327874 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 2.28242e+06us
I20260809 07:33:28.216271 2250 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.272367 1958 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.313153 2253 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.285537 2117 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.342222 2200 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.275847 31048 rpcz_store.cc:276] 0809 07:33:25.401325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.573095 (+171770us) service_pool.cc:227] Handling call
0809 07:33:25.601762 (+ 28667us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:25.601964 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:25.775602 (+173638us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.274929 (+2499327us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:28.274985 (+ 56us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:28.275032 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.275093 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.275381 (+ 288us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":847,"prepare.queue_time_us":51935,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":89,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":21329,"thread_start_us":2643105,"threads_started":2}
I20260809 07:33:28.351850 2150 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:28.326520 32141 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.267s user 0.009s sys 0.019s
W20260809 07:33:28.326841 1621 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.392s user 0.008s sys 0.018s
W20260809 07:33:28.410190 31005 connection.cc:232] RPC call timeout handler was delayed by -1.3070267280563712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260809 07:33:28.459645 31005 connection.cc:232] RPC call timeout handler was delayed by -1.0483666870277375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260809 07:33:28.460285 31005 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.617s (SENT) [suppressed 45 similar messages]
I20260809 07:33:28.464723 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.464986 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.465504 31174 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:28.466243 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.467083 31177 raft_consensus.cc:2396] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:28.467182 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:28.465690 31170 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:28.467823 31174 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:28.471033 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:28.456054 32141 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.397s user 0.009s sys 0.019s
W20260809 07:33:28.455951 1621 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.521s user 0.008s sys 0.018s
I20260809 07:33:28.485038 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.486166 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.489370 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.505237 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:28.509199 31006 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.087s user 0.014s sys 0.043s
W20260809 07:33:28.509584 31006 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.088s user 0.014s sys 0.043s
I20260809 07:33:28.500524 1898 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:28.507627 2019 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.709s user 0.042s sys 0.049s
W20260809 07:33:28.514088 2019 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.717s user 0.043s sys 0.049s
I20260809 07:33:28.513727 2250 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.507032 2200 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.515157 2253 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.515926 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
W20260809 07:33:28.513511 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 2.30749e+06us
I20260809 07:33:28.520372 2309 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.517577 1898 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.520925 2117 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.532742 2191 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.534955 1517 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.525095 1958 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.535148 2150 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.550765 31174 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:28.534063 2309 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.536100 1898 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:28.562274 2253 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.570405 2253 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:28.576591 32136 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.186s user 0.005s sys 0.059s
W20260809 07:33:28.587208 32136 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.241s user 0.005s sys 0.059s
I20260809 07:33:28.595326 1898 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:28.604756 2309 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.607089 2253 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:28.607748 2266 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.919s user 0.015s sys 0.010s
W20260809 07:33:28.608206 2266 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.947s user 0.015s sys 0.011s
W20260809 07:33:28.614761 2233 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s user 0.024s sys 0.041s
W20260809 07:33:28.615132 2233 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s user 0.024s sys 0.041s
I20260809 07:33:28.621300 2301 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.621872 2301 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:28.624241 2330 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.624845 2330 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:28.626910 2301 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.627556 2330 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:28.661849 31003 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.642s user 0.038s sys 0.047s
W20260809 07:33:28.662971 31003 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.794s user 0.039s sys 0.047s
I20260809 07:33:28.663122 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1223) took 3221 ms. Trace:
I20260809 07:33:28.663609 30890 rpcz_store.cc:276] 0809 07:33:25.441181 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.481879 (+1040698us) service_pool.cc:227] Handling call
0809 07:33:26.482303 (+ 424us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:26.482680 (+ 377us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.482709 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:26.483156 (+ 447us) log.cc:842] Serialized 26 byte log entry
0809 07:33:28.174804 (+1691648us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:28.174850 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.174932 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:28.662392 (+487460us) raft_consensus.cc:1671] finished
0809 07:33:28.662478 (+ 86us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.662877 (+ 399us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":193127,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":1691198,"threads_started":1,"wal-append.queue_time_us":2123751}
I20260809 07:33:28.666977 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1238) took 3026 ms. Trace:
I20260809 07:33:28.667452 30911 rpcz_store.cc:276] 0809 07:33:25.640746 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.793268 (+1152522us) service_pool.cc:227] Handling call
0809 07:33:26.802087 (+ 8819us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:26.802527 (+ 440us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.802557 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:26.834106 (+ 31549us) log.cc:842] Serialized 26 byte log entry
0809 07:33:28.472573 (+1638467us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:28.472616 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.472682 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:28.666346 (+193664us) raft_consensus.cc:1671] finished
0809 07:33:28.666391 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.666723 (+ 332us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13050,"thread_start_us":1431777,"threads_started":1,"wal-append.queue_time_us":1776889}
W20260809 07:33:28.674782 30853 connection.cc:232] RPC call timeout handler was delayed by -1.0896587781608105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260809 07:33:28.661273 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 204148us
I20260809 07:33:28.682933 30852 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:28.685213 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:28.686889 31054 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.286s user 0.045s sys 0.125s
W20260809 07:33:28.686925 31059 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.883s user 0.012s sys 0.075s
W20260809 07:33:28.687606 31059 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.884s user 0.013s sys 0.075s
I20260809 07:33:28.687429 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1356) took 3232 ms. Trace:
I20260809 07:33:28.687496 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.689052 30891 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 3.
I20260809 07:33:28.690929 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:28.691202 31058 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s user 0.006s sys 0.011s
W20260809 07:33:28.692451 31058 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s user 0.006s sys 0.011s
I20260809 07:33:28.694710 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:28.687517 31054 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.287s user 0.045s sys 0.126s
W20260809 07:33:28.696648 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.606s user 0.005s sys 0.002s
W20260809 07:33:28.697069 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.607s user 0.005s sys 0.002s
I20260809 07:33:28.698031 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.698832 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:28.699456 31120 connection.cc:232] RPC call timeout handler was delayed by -1.8220081380568445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260809 07:33:28.699905 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.710841 31164 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:28.712064 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.700378 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.700670 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:28.588726 1195 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 2.564s user 0.023s sys 0.068s
I20260809 07:33:28.701552 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.702919 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.704002 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.706633 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
W20260809 07:33:28.708611 31120 connection.cc:232] RPC call timeout handler was delayed by -1.0107380589470267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260809 07:33:28.736562 1898 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:28.728792 1195 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 2.708s user 0.023s sys 0.069s
I20260809 07:33:28.738763 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:28.740979 31048 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 6.
I20260809 07:33:28.741339 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:28.729101 31177 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:28.743988 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:28.744647 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:28.712006 1856 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s user 0.002s sys 0.010s
W20260809 07:33:28.746850 1856 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s user 0.002s sys 0.010s
I20260809 07:33:28.690851 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1250) took 3309 ms. Trace:
I20260809 07:33:28.737535 1898 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.731355 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:28.731986 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:28.730870 31174 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:28.733549 31170 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:28.731387 31181 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:28.688416 31051 rpcz_store.cc:276] 0809 07:33:25.454927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.473941 (+1019014us) service_pool.cc:227] Handling call
0809 07:33:26.494457 (+ 20516us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:26.532853 (+ 38396us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:26.532884 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:26.533367 (+ 483us) log.cc:842] Serialized 26 byte log entry
0809 07:33:28.189863 (+1656496us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:28.189904 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.189977 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:28.686837 (+496860us) raft_consensus.cc:1671] finished
0809 07:33:28.686875 (+ 38us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.687173 (+ 298us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":37957,"raft.queue_time_us":109290,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":12625,"thread_start_us":1656146,"threads_started":1,"wal-append.queue_time_us":2068402}
I20260809 07:33:28.743218 1517 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.750578 31055 rpcz_store.cc:276] 0809 07:33:25.381375 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.381519 (+ 144us) service_pool.cc:227] Handling call
0809 07:33:25.381867 (+ 348us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:25.583911 (+202044us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:25.584032 (+ 121us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:25.602719 (+ 18687us) log.cc:842] Serialized 26 byte log entry
0809 07:33:26.573492 (+970773us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b48000d1058. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:28.278114 (+1704622us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:28.278155 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.278225 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:28.690219 (+411994us) raft_consensus.cc:1671] finished
0809 07:33:28.690270 (+ 51us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.690655 (+ 385us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30249,"raft.queue_time_us":43398,"raft.run_cpu_time_us":1686,"raft.run_wall_time_us":1140648,"spinlock_wait_cycles":12174592,"thread_start_us":2809278,"threads_started":2,"wal-append.queue_time_us":3011616}
I20260809 07:33:28.755281 1517 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:28.759443 31169 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:28.812657 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.816181 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:28.818888 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.828194 2250 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.831449 2256 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.832080 2250 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.832692 2326 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:28.832460 2256 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.833504 2326 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.831547 2306 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.834579 2253 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:28.835042 2253 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:28.834754 2306 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.836743 2250 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.837973 2251 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.838757 2215 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.839207 1195 raft_consensus.cc:2741] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 07:33:28.840133 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1367) took 1924 ms. Trace:
W20260809 07:33:28.842412 31117 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.659s user 0.000s sys 0.012s
W20260809 07:33:28.851960 30909 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s user 0.005s sys 0.014s
W20260809 07:33:28.838790 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 378130us
I20260809 07:33:28.836882 2233 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:28.853307 2302 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.828263 2255 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:28.842772 2019 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.854817 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1263) took 3229 ms. Trace:
W20260809 07:33:28.858732 31117 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.709s user 0.000s sys 0.012s
W20260809 07:33:28.851713 30898 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s user 0.008s sys 0.016s
I20260809 07:33:28.839663 2283 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:28.828277 2191 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.861974 2233 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.862609 1517 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:28.863103 2213 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.863788 2353 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260809 07:33:28.834213 2311 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.861743 2096 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:28.859511 30890 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:28.854971 2363 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:28.864792 30909 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s user 0.005s sys 0.014s
I20260809 07:33:28.837459 2243 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:28.845517 2253 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:28.842690 2023 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:28.870612 30911 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:28.889272 30898 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s user 0.008s sys 0.016s
W20260809 07:33:28.895061 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 77038.6us
I20260809 07:33:28.896206 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.865020 2283 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:28.857347 31054 rpcz_store.cc:276] 0809 07:33:26.915337 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.234675 (+319338us) service_pool.cc:227] Handling call
0809 07:33:27.334302 (+ 99627us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.334559 (+ 257us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:27.400593 (+ 66034us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.813261 (+1412668us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:28.821906 (+ 8645us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:28.831194 (+ 9288us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.831301 (+ 107us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.840032 (+ 8731us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":74360,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":157592,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":269,"thread_start_us":1402619,"threads_started":2,"wal-append.queue_time_us":1225348}
I20260809 07:33:28.861238 31059 rpcz_store.cc:276] 0809 07:33:25.625795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.757260 (+1131465us) service_pool.cc:227] Handling call
0809 07:33:26.757618 (+ 358us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:26.757834 (+ 216us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:26.803191 (+ 45357us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.853471 (+2050280us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.854037 (+ 566us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.854202 (+ 165us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:28.854289 (+ 87us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:28.854414 (+ 125us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.854520 (+ 106us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.854757 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":48117,"prepare.run_cpu_time_us":2168,"prepare.run_wall_time_us":11204,"raft.queue_time_us":162450,"thread_start_us":2018219,"threads_started":2,"wal-append.queue_time_us":1797360}
I20260809 07:33:28.935230 2361 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.936221 2361 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.936482 2311 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.935268 2357 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:28.938526 2008 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:28.938803 2357 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.939318 2008 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.939810 2311 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.940740 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1222) took 3503 ms. Trace:
I20260809 07:33:28.941282 2357 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.941505 2251 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.933470 2191 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.943975 2251 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.944169 2191 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.945111 2213 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.946705 2361 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.947765 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:28.948577 2326 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.949249 1898 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:28.942437 2310 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260809 07:33:28.951308 2363 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.948640 32198 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:28.954531 2019 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.957028 2019 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.958022 2243 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.960409 2243 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.961211 2302 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.928099 2283 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:28.961212 30904 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:28.950037 2256 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.953713 2363 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.986428 2096 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:28.936249 2150 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.063s
I20260809 07:33:29.009752 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.987444 2326 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:28.989243 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.991586 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1363) took 3236 ms. Trace:
W20260809 07:33:28.995129 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 60861.3us
I20260809 07:33:28.987474 2342 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
W20260809 07:33:28.996578 2364 log.cc:924] Time spent T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260809 07:33:28.941126 30896 rpcz_store.cc:276] 0809 07:33:25.440883 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:26.068192 (+627309us) service_pool.cc:227] Handling call
0809 07:33:26.068553 (+ 361us) raft_consensus.cc:1457] Updating replica for 3 ops
0809 07:33:28.248424 (+2179871us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:28.248457 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 3 ops
0809 07:33:28.273637 (+ 25180us) log.cc:842] Serialized 78 byte log entry
0809 07:33:28.853851 (+580214us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:28.854485 (+ 634us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.924691 (+ 70206us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.934731 (+ 10040us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.934916 (+ 185us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:28.940354 (+ 5438us) raft_consensus.cc:1671] finished
0809 07:33:28.940373 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.940589 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21493,"prepare.queue_time_us":1862,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":99,"raft.queue_time_us":627936,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":2682009,"threads_started":2,"wal-append.queue_time_us":562625}
W20260809 07:33:29.012396 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 75231.1us
I20260809 07:33:29.011276 31058 rpcz_store.cc:276] 0809 07:33:25.755503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:27.193671 (+1438168us) service_pool.cc:227] Handling call
0809 07:33:27.520748 (+327077us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:27.854526 (+333778us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:27.885614 (+ 31088us) log.cc:842] Serialized 14 byte log entry
0809 07:33:28.814504 (+928890us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:28.814604 (+ 100us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:28.814739 (+ 135us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:28.991241 (+176502us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54002ef9d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:28.991295 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:28.991514 (+ 219us) inbound_call.cc:173] Queueing success response
0809 07:33:28.994966 (+ 3452us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b48000d1058. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
Metrics: {"prepare.queue_time_us":30920,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":159161,"raft.run_cpu_time_us":3254,"raft.run_wall_time_us":138810,"spinlock_wait_cycles":737920768,"thread_start_us":924346,"threads_started":2,"wal-append.queue_time_us":729283}
I20260809 07:33:29.015960 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:28.983757 2357 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.022166 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.022838 2023 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.025761 2023 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.029346 30900 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.022739 2357 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.035005 2357 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.036257 2306 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.037609 2213 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.006104 2311 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:29.038834 2311 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.039446 2302 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.041006 2311 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.042973 1621 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:29.043691 1621 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.048529 2233 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.981680 2008 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:28.941115 2255 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.054746 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.055593 31048 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:28.950135 2215 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.057190 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.057828 31058 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:29.056345 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.059671 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.060479 31054 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 3.
I20260809 07:33:29.061801 30911 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:29.021876 2326 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.058110 30906 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.068359 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.069339 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.085906 1898 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:29.086817 1898 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.089284 2096 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.096114 2310 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:29.096551 2310 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.096875 2310 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:29.098263 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.098984 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:29.099857 31054 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:29.099936 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.100658 30896 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:29.101064 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.101950 31054 raft_consensus.cc:2396] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 07:33:29.102557 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:29.103129 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.103407 30906 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:29.103793 31054 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:29.104508 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.105413 30909 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:29.105873 1898 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.106411 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.096252 2326 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.109617 31165 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260809 07:33:29.115669 2255 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.112491 30741 catalog_manager.cc:5495] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: config changed from index 2 to 5, 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:29.120975 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.119128 2283 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.125908 30904 raft_consensus.cc:2354] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:29.126137 2215 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.127804 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.121784 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.129650 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.130380 1621 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.130756 2339 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:29.131106 1621 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:29.131263 2339 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:29.131570 1621 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.131739 2339 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:29.132009 1621 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:29.133369 2283 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.140841 30890 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:29.136612 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.141532 31059 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 5.
I20260809 07:33:29.137396 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.134627 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.140739 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.143080 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:29.143193 31051 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:29.143784 1856 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:29.145694 31058 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:29.145344 1856 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.146144 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.176923 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.180303 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.181056 30890 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:29.181694 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.181830 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.181998 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.183017 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.183799 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.183995 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:29.184572 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.189417 31164 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:29.185981 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.186750 2096 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.186779 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.187183 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.178586 30899 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:29.194854 31181 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.195863 31051 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:29.198611 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.199962 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.200394 2150 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:29.201985 2150 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:29.202551 2150 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:29.176633 2255 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.203366 2255 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.203532 2150 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.205606 2255 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.206696 2096 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.208222 31120 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.208443 1856 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.208832 2096 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.209240 2371 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:29.209599 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.210767 2372 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:29.211324 2372 raft_consensus.cc:686] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:29.212396 2372 consensus_queue.cc:237] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.214659 30909 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:29.220041 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:29.199802 1898 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:29.226472 1898 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.226883 1898 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.227020 2096 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:29.227664 2096 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.229532 2096 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.229967 2096 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.230475 2096 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.230727 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.231470 31051 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:29.232383 2096 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.233636 31059 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260809 07:33:29.233986 31059 pending_rounds.cc:76] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 4
I20260809 07:33:29.234191 31059 pending_rounds.cc:97] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260809 07:33:29.234365 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.235162 31058 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:29.236076 2371 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:29.236673 2371 raft_consensus.cc:686] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:29.234453 31059 raft_consensus.cc:2866] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:29.238632 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.239344 31048 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:29.241823 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.242547 31048 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:29.224917 2379 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:29.225452 2255 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.144618 2310 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.271024 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.271821 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.272723 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.282536 31181 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:33:29.290766 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 67456.2us
I20260809 07:33:29.237763 2371 consensus_queue.cc:237] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.325915 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.320201 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:29.302747 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.297905 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:29.298643 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.304373 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:29.369220 31054 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:29.380524 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 76817.1us
I20260809 07:33:29.291960 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.428592 31175 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:29.429584 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.430228 31175 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.304936 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.431095 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.431691 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.432196 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.432932 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.433732 2251 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:29.326182 2253 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.434332 2251 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.435190 31059 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:29.435797 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.435320 31170 raft_consensus.cc:2354] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:29.434653 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.436525 2251 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.436864 31048 raft_consensus.cc:2396] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260809 07:33:29.437676 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.438306 31048 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:29.439208 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:29.439819 31048 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 6
I20260809 07:33:29.440874 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.437690 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.434048 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.432945 31175 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 4.
I20260809 07:33:29.442237 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.397240 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.369861 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.307516 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.436120 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.453708 31177 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.456655 31059 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:29.306888 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.439680 2253 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.477088 31049 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.493589 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:29.477916 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.389051 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.455384 31057 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:29.496060 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.496681 31175 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:29.494575 31054 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.456161 31046 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:29.490446 31165 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.495543 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:29.501142 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.498306 31169 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:29.375166 2310 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.500975 30852 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.504930 30899 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 5.
I20260809 07:33:29.369393 2255 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.390202 2339 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.509895 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.383584 1621 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.531486 31059 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
W20260809 07:33:29.532604 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 54936.9us
I20260809 07:33:29.497978 2251 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:29.564370 30852 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.570729 31172 raft_consensus.cc:2354] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:29.372803 2379 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.577555 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.457974 1898 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.583542 31045 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:29.576756 1856 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:29.622426 31182 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:29.624922 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.629354 31059 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:29.629630 31120 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:29.625655 31057 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:29.632928 2255 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.633176 1898 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.631341 31049 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
W20260809 07:33:29.636606 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 457053us
I20260809 07:33:29.636899 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.639925 1621 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.641151 31055 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260809 07:33:29.641273 2384 log.cc:924] Time spent T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260809 07:33:29.641775 30852 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.642040 31055 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:36496
I20260809 07:33:29.642385 31049 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:29.642812 31055 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:33:29.643849 2243 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 07:33:29.644222 2243 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.644574 2243 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:29.646409 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.637144 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.643324 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.643242 31055 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:29.647905 31049 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:29.652639 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.657420 30853 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:29.626616 1856 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.630069 2251 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:29.669216 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:29.676591 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 74676.6us
I20260809 07:33:29.677954 2379 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.671485 1856 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.689556 31054 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:29.679147 2283 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:29.696923 31045 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:29.695638 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.675225 1621 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:29.697860 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.695015 2309 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:29.709578 31048 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:29.669335 2251 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.721642 31059 raft_consensus.cc:2396] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260809 07:33:29.725199 2310 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:29.726927 31120 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.726850 1621 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:29.699260 2283 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:29.754204 2310 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:29.755044 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.756578 2353 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:29.757128 2353 raft_consensus.cc:686] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:29.759662 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:29.733641 2251 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:29.715914 2191 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260809 07:33:29.749567 2283 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.755381 2310 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.758675 2339 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.778851 31049 raft_consensus.cc:1261] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:29.778541 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:29.787153 31057 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 6
I20260809 07:33:29.794418 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.783463 2379 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:29.795961 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.801877 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.802103 31169 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:29.804056 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.774780 1898 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260809 07:33:29.758296 2353 consensus_queue.cc:237] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.806908 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:29.805439 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 159191us
I20260809 07:33:29.809492 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.809432 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.810017 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.810153 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.789716 1621 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260809 07:33:29.798666 2379 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.813690 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.815994 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.822932 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.823117 30852 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.823426 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.825582 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.826586 31174 raft_consensus.cc:2396] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 07:33:29.827086 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.824357 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.817597 2379 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.829687 31181 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:29.815757 2349 consensus_queue.cc:1035] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260809 07:33:29.834580 30852 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.835022 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:29.839609 31165 raft_consensus.cc:2396] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260809 07:33:29.852562 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 195323us
I20260809 07:33:29.858799 30853 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:29.876305 1856 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.880513 2415 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:29.881326 2415 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:29.881667 2415 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.881982 2415 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:29.882010 1856 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.882431 2309 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:29.882892 2309 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:29.883749 1856 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:29.884191 1856 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.884537 1856 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.883802 2309 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.893772 31058 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
W20260809 07:33:29.909359 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 201608us
I20260809 07:33:29.901433 2309 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:29.924163 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 154256us
I20260809 07:33:29.927989 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40566 (request call id 152) took 8165 ms. Trace:
I20260809 07:33:29.931756 2417 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:29.933552 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.934392 2315 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:29.934775 2315 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:29.935115 2315 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:29.935809 2415 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.937963 2415 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.942831 2315 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.945272 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.947377 1621 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:29.947786 1621 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:29.948176 1621 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:29.948586 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.949730 1517 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:29.949847 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.951123 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:29.951447 2371 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:29.946525 2315 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.951900 2371 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:29.952309 2371 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:29.952446 2253 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:29.952962 2253 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:29.953320 2253 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:29.958287 1517 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:29.958766 1517 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:29.963902 31055 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.964978 31051 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 10: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:29.971637 31055 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.972072 2243 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.974247 2283 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:29.975386 31006 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:29.976286 2283 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.976773 31006 leader_election.cc:398] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:29.977097 31006 leader_election.cc:401] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:29.977706 2255 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:29.978164 2255 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:29.978564 2255 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:29.974092 2243 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:29.981109 31006 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:29.997725 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 164259us
I20260809 07:33:29.963672 2253 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.012482 31006 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:30.013823 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 62768us
W20260809 07:33:30.025043 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 168064us
I20260809 07:33:30.032546 2251 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.033015 2371 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.038431 2382 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:30.038952 2382 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:30.039335 2382 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:30.047834 2253 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:29.930025 30745 rpcz_store.cc:276] 0809 07:33:21.684449 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:21.692003 (+ 7554us) service_pool.cc:227] Handling call
0809 07:33:29.849556 (+8157553us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:25.225238 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:25.291460 (+ 66222us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:25.291499 (+ 39us) write_op.cc:435] Acquired schema lock
0809 07:33:25.291516 (+ 17us) tablet.cc:660] Decoding operations
0809 07:33:25.391927 (+100411us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:25.392014 (+ 87us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:25.392099 (+ 85us) tablet.cc:683] Acquiring locks for 27 operations
0809 07:33:25.816532 (+424433us) tablet.cc:699] Row locks acquired
0809 07:33:25.816560 (+ 28us) write_op.cc:260] PREPARE: finished
0809 07:33:25.816712 (+ 152us) write_op.cc:270] Start()
0809 07:33:25.816824 (+ 112us) write_op.cc:276] Timestamp: P: 1786260805816661 usec, L: 0
0809 07:33:25.816843 (+ 19us) op_driver.cc:331] REPLICATION: starting
0809 07:33:25.817585 (+ 742us) log.cc:842] Serialized 10601 byte log entry
0809 07:33:28.604231 (+2786646us) op_driver.cc:444] REPLICATION: finished
0809 07:33:28.990993 (+386762us) write_op.cc:301] APPLY: starting
0809 07:33:28.991184 (+ 191us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:28.992042 (+ 858us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:28.992206 (+ 164us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:29.001294 (+ 9088us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:29.001852 (+ 558us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=27
0809 07:33:29.001925 (+ 73us) write_op.cc:312] APPLY: finished
0809 07:33:29.002657 (+ 732us) log.cc:842] Serialized 181 byte log entry
0809 07:33:29.002948 (+ 291us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:29.003680 (+ 732us) write_op.cc:454] Released schema lock
0809 07:33:29.003769 (+ 89us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:29.004230 (+ 461us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:29.004262 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":50030,"reported_tablets":57,"child_traces":[["op",{"apply.queue_time_us":386285,"apply.run_cpu_time_us":8911,"apply.run_wall_time_us":115500,"num_ops":27,"prepare.queue_time_us":383657,"prepare.run_cpu_time_us":13546,"prepare.run_wall_time_us":658235,"raft.queue_time_us":1283116,"raft.run_cpu_time_us":44380,"raft.run_wall_time_us":561607,"replication_time_us":2787288,"thread_start_us":1805827,"threads_started":5,"wal-append.queue_time_us":669112}]]}
I20260809 07:33:30.046062 31057 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:30.062743 30853 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.061955 30852 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.045753 2379 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:30.072656 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 45851.9us
W20260809 07:33:30.072731 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46810.3us
I20260809 07:33:30.066298 2326 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6.
I20260809 07:33:30.075722 2326 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.076069 2326 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:30.076550 2326 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.078503 2427 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:30.079190 2427 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.079527 2427 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.079829 2427 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.085585 1621 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.090075 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.089079 30852 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.081363 1517 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:30.096787 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 86262.3us
I20260809 07:33:30.095165 2251 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.103413 30852 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.104126 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.104697 2430 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.107940 2430 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.108434 2430 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.101811 2243 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 07:33:30.110687 2243 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.111143 2243 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:30.106033 1856 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.108270 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.113813 1856 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.105870 2371 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:30.117120 1621 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:30.122514 30891 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:30.126441 2255 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.127481 2382 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.128412 2255 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.132552 30853 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.133499 2283 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.133797 32198 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 07:33:30.133894 2283 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.134444 2283 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.136390 2441 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 07:33:30.137106 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.138020 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.138752 30890 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:30.138964 30896 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260809 07:33:30.139708 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.140255 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.140821 30909 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:30.141551 2283 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.144358 1517 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:30.145294 2326 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.129369 2382 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.146860 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:30.146016 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 43977us
I20260809 07:33:30.127413 2379 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.151965 2283 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.155583 30741 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 25) took 6045 ms. Trace:
I20260809 07:33:30.156675 2430 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.158244 2427 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.155956 30741 rpcz_store.cc:276] 0809 07:33:24.109928 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:24.110100 (+ 172us) service_pool.cc:227] Handling call
0809 07:33:30.155491 (+6045391us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:29.568824 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:29.642859 (+ 74035us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:29.642915 (+ 56us) write_op.cc:435] Acquired schema lock
0809 07:33:29.642945 (+ 30us) tablet.cc:660] Decoding operations
0809 07:33:29.643402 (+ 457us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:29.643482 (+ 80us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:29.643514 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:33:29.643798 (+ 284us) tablet.cc:699] Row locks acquired
0809 07:33:29.643831 (+ 33us) write_op.cc:260] PREPARE: finished
0809 07:33:29.643992 (+ 161us) write_op.cc:270] Start()
0809 07:33:29.644137 (+ 145us) write_op.cc:276] Timestamp: P: 1786260809643933 usec, L: 0
0809 07:33:29.644157 (+ 20us) op_driver.cc:331] REPLICATION: starting
0809 07:33:29.644808 (+ 651us) log.cc:842] Serialized 594 byte log entry
0809 07:33:29.963545 (+318737us) op_driver.cc:444] REPLICATION: finished
0809 07:33:30.101952 (+138407us) write_op.cc:301] APPLY: starting
0809 07:33:30.106137 (+ 4185us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:30.138554 (+ 32417us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:30.138679 (+ 125us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:30.138899 (+ 220us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:30.139055 (+ 156us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:33:30.139100 (+ 45us) write_op.cc:312] APPLY: finished
0809 07:33:30.139282 (+ 182us) log.cc:842] Serialized 22 byte log entry
0809 07:33:30.139441 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:30.139530 (+ 89us) write_op.cc:454] Released schema lock
0809 07:33:30.143851 (+ 4321us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:30.144523 (+ 672us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:30.144556 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":130868,"apply.run_cpu_time_us":1947,"apply.run_wall_time_us":46056,"num_ops":1,"prepare.queue_time_us":385986,"prepare.run_cpu_time_us":3191,"prepare.run_wall_time_us":114426,"replication_time_us":284613,"thread_start_us":161083,"threads_started":2}]]}
I20260809 07:33:30.160349 2427 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.162246 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.162526 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.162917 30900 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:30.163085 30909 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:30.165443 30896 raft_consensus.cc:1226] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:30.161774 2430 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:30.168021 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 64665.1us
I20260809 07:33:30.168646 31006 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff
I20260809 07:33:30.169770 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.170951 31006 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.171830 31006 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.172353 2382 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:30.173120 2243 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.174126 2379 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.173189 2382 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:30.168749 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.178622 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.179024 30896 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:30.179430 2382 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.178519 2379 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:30.179157 2447 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.180107 2379 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.175787 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.177096 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:30.180985 31182 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 6.
I20260809 07:33:30.181087 2243 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.177718 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.180452 2447 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.186592 2447 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.189523 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.181124 31006 leader_election.cc:398] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:30.191107 30744 catalog_manager.cc:5495] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.187521 2326 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.192411 32141 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.190747 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:30.191220 31006 leader_election.cc:401] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260809 07:33:30.194432 30852 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.194336 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:30.195734 2446 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.179917 2382 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:30.187803 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.177561 2427 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.199756 30852 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.200937 2430 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.193428 30744 catalog_manager.cc:5495] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:30.203027 2430 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.204782 2283 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.205283 30896 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:30.205368 31006 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.205168 2283 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:30.203457 2430 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.207624 2283 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.203583 2427 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.208942 2427 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.209260 2326 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:30.207593 2441 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.211504 2441 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:30.211889 2441 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.198179 2243 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.214674 2243 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.219045 30896 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:30.220431 2243 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.221783 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.221468 30909 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 07:33:30.220655 2446 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:30.224412 1856 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 07:33:30.224784 1856 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.225114 1856 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:30.225276 30896 raft_consensus.cc:1226] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:30.200033 2251 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:30.227735 2309 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:33:30.228303 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:30.229107 2353 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.229452 2353 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:30.229828 2353 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.230871 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.232935 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.233955 30909 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:30.235002 2251 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.235455 2251 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:30.236086 2251 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.238241 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:30.207141 1517 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:30.246644 1517 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:30.247035 1517 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:30.246873 2353 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.242642 30744 catalog_manager.cc:5495] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.218007 2283 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.198366 2379 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.283996 30744 catalog_manager.cc:5495] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 6, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:33:30.300879 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 92052.2us
I20260809 07:33:30.300948 30853 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.247129 2382 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:30.250998 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 58841.7us
W20260809 07:33:30.335608 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 48473.2us
W20260809 07:33:30.328217 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 92823.3us
I20260809 07:33:30.336342 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:30.346176 30906 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:30.342486 30744 catalog_manager.cc:5495] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:30.340412 2251 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.367875 2251 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.372416 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.376341 30898 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:30.376765 30911 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:30.378480 30744 catalog_manager.cc:5495] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.327291 2449 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:30.382436 31006 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.302119 2447 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.375489 2283 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:30.396864 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 146747us
I20260809 07:33:30.378509 2353 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:30.399154 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.383225 2449 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:30.408700 30911 raft_consensus.cc:2379] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:30.375986 2430 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.371929 2441 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.405408 2326 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.418098 30744 catalog_manager.cc:5495] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 4 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.421895 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.408581 2449 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.397836 32141 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.420511 2283 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260809 07:33:30.655135 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:30.649190 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:30.646131 30904 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:30.544344 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.426410 30911 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
W20260809 07:33:30.489930 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47459.9us
I20260809 07:33:30.677174 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.677673 31182 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.678987 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.679384 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:30.679898 31170 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:30.442379 2459 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:30.680608 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.430414 2447 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.681303 31164 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:30.432240 2243 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.419425 2326 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.682574 2326 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.646024 2430 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.646765 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.683773 30899 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:30.684396 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.684774 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.685031 31046 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:30.685890 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.686651 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.687716 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.687914 2243 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.688234 30904 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.431705 1856 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.692210 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.692795 31046 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.693076 1856 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.491729 2379 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.646531 2446 raft_consensus.cc:3032] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:30.505766 2417 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 07:33:30.645448 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 224479us
I20260809 07:33:30.699717 2459 raft_consensus.cc:686] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:30.697422 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.701190 2309 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.701620 2309 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.701963 2309 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.702405 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.702579 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.703766 30899 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:30.704690 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.705240 31058 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:30.697492 2446 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:30.708982 2446 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.700625 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:30.709935 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 54901.2us
I20260809 07:33:30.700757 2459 consensus_queue.cc:237] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.724774 31058 raft_consensus.cc:2454] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 4.
I20260809 07:33:30.725003 31164 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:30.710741 2382 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.713774 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.762071 31057 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.717123 31046 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:30.719393 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.763077 31048 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 5: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:30.766888 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.767210 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:30.585041 2453 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.164s user 0.000s sys 0.000s
I20260809 07:33:30.426651 30744 catalog_manager.cc:5495] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 4 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.667054 30898 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:30.714484 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.729846 30904 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:30.729398 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.739462 2309 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.728125 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.769891 31045 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.733245 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:30.726573 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.770617 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:30.732762 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.771891 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.772629 31174 raft_consensus.cc:2379] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:30.725932 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.773661 31058 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:30.771356 30911 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.775893 31046 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:30.733893 30899 raft_consensus.cc:1261] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 07:33:30.728730 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.777398 31054 raft_consensus.cc:2379] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:30.775249 31058 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:30.722947 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:30.728953 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.783186 31170 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:30.731328 31165 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 07:33:30.768636 31055 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:30.769150 31172 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.738587 2430 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:30.728669 2326 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.738703 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.770927 31051 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.783946 30898 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260809 07:33:30.785434 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 72849.4us
I20260809 07:33:30.421387 2427 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.787441 30904 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:30.838423 31048 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:30.844010 31049 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.814502 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.763943 2441 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.433674 1517 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.851539 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.852991 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:30.854692 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 121984us
I20260809 07:33:30.855355 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:30.856127 31005 leader_election.cc:334] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.617s (SENT)
I20260809 07:33:30.859488 30899 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260809 07:33:30.857194 30744 catalog_manager.cc:5495] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:30.861756 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.786835 2353 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:30.863189 31046 raft_consensus.cc:2379] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 9: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:30.863765 30891 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:30.861699 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.865397 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.867599 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.839823 2417 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:30.868312 31046 raft_consensus.cc:2354] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8.
I20260809 07:33:30.867784 30898 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:30.846509 2243 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:30.869823 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.866559 30904 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.872911 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.845497 2415 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:30.876428 30853 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:30.877357 30852 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.881531 31169 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:30.887108 31049 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:30.887650 31055 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:30.889876 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.890450 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.890659 31055 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:30.857170 2446 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:30.892208 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.901188 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:30.902639 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 46553.1us
I20260809 07:33:30.904116 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.858474 2430 raft_consensus.cc:686] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:30.904805 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.866297 2353 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:30.862490 2457 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:30.904776 31058 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:30.909205 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.927908 31054 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:30.928846 30898 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:30.928395 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.898211 30744 catalog_manager.cc:5495] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.917407 31057 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:30.920358 31182 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:30.920902 31045 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:30.890470 2458 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:30.925475 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.874821 2251 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:30.882856 2243 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:30.896018 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.910946 31046 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:30.929775 31048 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:30.930939 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.926069 30904 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:30.930979 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.936707 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.938282 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:30.926702 2353 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:30.939653 31045 raft_consensus.cc:2379] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:30.941866 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 72972.3us
I20260809 07:33:30.939657 31054 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:30.946962 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:30.947747 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 70495.1us
I20260809 07:33:30.948985 30852 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:30.945897 31057 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
W20260809 07:33:30.920673 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 521018us
I20260809 07:33:30.932937 2447 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:30.952913 31054 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
W20260809 07:33:30.953616 32139 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 339027us
I20260809 07:33:30.955089 31059 pending_rounds.cc:76] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 4
I20260809 07:33:30.965690 31059 pending_rounds.cc:97] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260809 07:33:30.966018 31059 raft_consensus.cc:2866] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:30.930294 1856 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:30.930449 2461 raft_consensus.cc:2781] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 07:33:30.888887 2415 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.968690 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:30.976590 31057 raft_consensus.cc:1226] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 3. Current term is 4. Ops: []
I20260809 07:33:30.976712 30744 catalog_manager.cc:5495] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:30.964068 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:30.962370 31058 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 11.
I20260809 07:33:30.896677 2417 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.938684 2243 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:30.936678 2459 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:30.983649 31045 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260809 07:33:30.981767 31046 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:30.988981 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 87897us
I20260809 07:33:30.976347 2426 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:31.000926 30853 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
W20260809 07:33:31.002351 2467 log.cc:924] Time spent T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260809 07:33:31.018014 31170 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
W20260809 07:33:31.003162 2466 log.cc:924] Time spent T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260809 07:33:31.028020 2251 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:31.028456 2251 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:31.028822 2251 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:31.003762 2430 consensus_queue.cc:237] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.005622 31051 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:30.941989 1621 consensus_queue.cc:1046] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260809 07:33:30.931841 2457 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:30.987150 2474 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:31.009539 30911 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:30.987502 30744 catalog_manager.cc:5495] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 6 to 7, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 7 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.010804 1517 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:30.987022 2461 raft_consensus.cc:686] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.042701 31051 raft_consensus.cc:1261] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 07:33:31.035265 2426 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.025520 2326 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:30.434772 2452 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:30.969425 2447 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.026521 30853 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:31.034472 31046 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:31.058609 30744 catalog_manager.cc:5495] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 7 to 8, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 8 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.036388 31045 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:31.008852 2427 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.026160 2309 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:30.988072 2459 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:31.050540 2482 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:31.051446 2461 consensus_queue.cc:237] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:31.082589 2471 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.145s user 0.000s sys 0.001s
I20260809 07:33:31.083122 2426 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.085150 2458 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:31.046060 1517 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:31.067888 31048 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:31.068207 31049 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:31.071529 2283 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:31.071918 32199 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
W20260809 07:33:31.074256 2470 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.186s user 0.000s sys 0.002s
I20260809 07:33:31.079160 2417 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.081053 2415 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.081290 31057 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:31.020206 31172 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:31.070205 30852 leader_election.cc:398] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:31.127769 30744 catalog_manager.cc:5495] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.137856 30852 leader_election.cc:401] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260809 07:33:31.126845 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.160840 30744 catalog_manager.cc:5495] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 5, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.173030 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:31.173720 31057 raft_consensus.cc:2362] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6. Re-sending same reply.
I20260809 07:33:31.095885 2458 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:31.175743 30744 catalog_manager.cc:5495] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 4 to 7, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67). New cstate: current_term: 7 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.134857 2283 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:31.099365 1517 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:31.087332 2459 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:31.187285 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 162633us
I20260809 07:33:31.183465 2283 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:31.195371 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.199369 31049 raft_consensus.cc:2379] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:31.196691 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.205835 30852 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.186018 1517 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:31.209457 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 183021us
I20260809 07:33:31.209661 30744 catalog_manager.cc:5495] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:31.148712 2449 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:31.197932 2417 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:31.214533 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.212713 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.216876 31046 raft_consensus.cc:2362] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8. Re-sending same reply.
I20260809 07:33:31.220240 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.212368 2483 consensus_queue.cc:1046] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260809 07:33:31.225409 2485 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:31.206007 1856 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260809 07:33:31.228897 1856 raft_consensus.cc:3032] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:31.229243 1856 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.230060 1856 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.231309 1856 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:31.234508 32198 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
W20260809 07:33:31.235706 31005 leader_election.cc:334] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.953s (SENT)
I20260809 07:33:31.236171 31005 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:31.237680 2449 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.239851 2326 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:31.240506 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.241096 31059 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:31.237944 31005 leader_election.cc:334] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.980s (SENT)
I20260809 07:33:31.247243 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:31.250337 30744 catalog_manager.cc:5495] T f25fc40767244ba0873588714d88e8b6 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.252249 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.220866 31046 raft_consensus.cc:2362] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6. Re-sending same reply.
W20260809 07:33:31.253649 31005 leader_election.cc:334] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 2.007s (SENT)
I20260809 07:33:31.254106 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:31.254136 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:31.257541 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.223501 2457 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.265020 31059 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 7.
I20260809 07:33:31.265508 31048 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:31.258641 31057 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:31.266729 31048 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:31.217259 2415 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260809 07:33:31.268837 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.275072 31172 raft_consensus.cc:3037] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:31.269554 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.275648 31170 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:31.270375 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 303741us
I20260809 07:33:31.276310 31046 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:31.290934 31057 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.249162 2446 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:31.294812 31046 raft_consensus.cc:1261] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:31.256184 2427 raft_consensus.cc:3032] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:31.261966 2326 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:31.302548 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.303481 31170 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:31.245415 2353 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.308770 31182 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.368314 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.371351 31169 raft_consensus.cc:2379] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
W20260809 07:33:31.370848 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 124710us
W20260809 07:33:31.374773 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 72294.4us
I20260809 07:33:31.376550 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:31.421944 31172 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:31.425794 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.375936 30744 catalog_manager.cc:5495] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.426895 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.427826 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.428736 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.429557 30890 raft_consensus.cc:2379] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:31.429410 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.427325 30744 catalog_manager.cc:5495] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:31.305224 2427 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.437081 31172 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.433063 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.448818 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.450178 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.450660 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.451709 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.452019 31170 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.452987 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.453398 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.454583 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.455256 31165 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:31.456962 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.457667 31172 raft_consensus.cc:2379] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:31.458829 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.460382 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.462399 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.462350 31165 raft_consensus.cc:2379] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:31.463987 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.466307 31170 raft_consensus.cc:2379] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:31.394783 1621 consensus_queue.cc:1046] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260809 07:33:31.395622 2382 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:31.396978 2243 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.397591 2309 consensus_queue.cc:1046] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260809 07:33:31.431351 2485 raft_consensus.cc:686] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.308451 2493 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.186s
I20260809 07:33:31.445174 2447 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.467227 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.488608 30890 raft_consensus.cc:2354] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:31.489435 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.490680 30904 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:31.491150 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.467716 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.466954 31172 raft_consensus.cc:2379] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 10: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:31.472359 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.467392 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.472136 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.493680 30909 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:31.479492 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.482503 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:31.447151 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 241342us
I20260809 07:33:31.472568 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.500496 31169 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:31.500396 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.503791 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.507647 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:31.468904 2490 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.066s user 0.000s sys 0.001s
I20260809 07:33:31.510232 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.499138 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:31.498723 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 122254us
I20260809 07:33:31.511669 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.512403 30909 raft_consensus.cc:2379] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:31.489956 2251 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.499332 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.499320 30852 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.479139 2483 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:31.514854 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.480806 2488 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:31.484014 1621 raft_consensus.cc:3032] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:31.515589 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.519586 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.519459 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.472905 2243 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.483073 2493 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:31.470131 2482 raft_consensus.cc:3032] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:31.524814 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.487142 2459 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.527985 2459 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.516165 31172 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.518829 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.520218 30899 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:31.480010 2353 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:31.522303 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.529390 30898 raft_consensus.cc:2379] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:31.523532 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.469761 2283 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:31.482203 2485 consensus_queue.cc:237] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.481819 2427 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.551451 30900 raft_consensus.cc:2396] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260809 07:33:31.609902 30896 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:31.521284 2488 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:31.592204 2514 raft_consensus.cc:3032] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 07:33:31.611853 2514 raft_consensus.cc:729] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:31.524422 2493 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:31.612955 2514 consensus_queue.cc:260] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.613243 2493 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.614365 2514 raft_consensus.cc:3037] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 3 FOLLOWER]: Advancing to term 4
I20260809 07:33:31.570822 2243 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:31.615294 2309 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:31.615878 30852 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.522646 1621 raft_consensus.cc:729] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:31.616616 2353 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:31.524066 2417 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:31.617084 2353 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:31.550932 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.551116 2482 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:31.611361 2488 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:31.618034 2353 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.600363 2251 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:31.553433 30909 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.620834 1621 consensus_queue.cc:260] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.622200 1621 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:31.623665 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:31.627372 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 213194us
I20260809 07:33:31.628901 31006 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.629992 2382 raft_consensus.cc:686] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:31.630924 2382 consensus_queue.cc:237] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.518833 2483 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:31.669672 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53897.2us
I20260809 07:33:31.673728 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:31.685189 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 164241us
I20260809 07:33:31.688743 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.689467 30891 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:31.611966 2283 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:31.716907 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.663129 2493 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:31.720201 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 46573.6us
I20260809 07:33:31.669556 2465 raft_consensus.cc:2781] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election won for term 8
W20260809 07:33:31.733785 31004 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 107827us
I20260809 07:33:31.672871 2243 raft_consensus.cc:686] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.722640 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.670193 2379 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260809 07:33:31.605803 1517 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:31.690631 2459 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:31.799561 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.810685 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:31.673575 2488 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.816937 2353 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:31.818099 2488 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:31.818687 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:31.784493 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 311226us
I20260809 07:33:31.818548 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.819823 31172 raft_consensus.cc:2354] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8.
I20260809 07:33:31.734156 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.820300 30852 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.819792 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:31.767751 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 138917us
I20260809 07:33:31.782650 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.821914 31006 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.822055 30890 raft_consensus.cc:2396] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260809 07:33:31.785852 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.823464 31006 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.823498 30904 raft_consensus.cc:2354] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:31.825163 31006 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.721477 2251 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:31.826087 2527 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 07:33:31.827777 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.828859 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.829653 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.830238 30899 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
W20260809 07:33:31.783722 2512 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260809 07:33:31.864094 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.864377 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.778246 2417 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:31.865391 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.865588 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.767825 2465 raft_consensus.cc:686] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:31.866916 2465 consensus_queue.cc:237] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.868324 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.869014 31172 raft_consensus.cc:2354] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
W20260809 07:33:31.869817 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 49600.6us
I20260809 07:33:31.870502 30852 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.870883 31182 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:31.830634 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.869858 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.865283 2417 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.873243 30904 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:31.874030 31170 raft_consensus.cc:2354] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
W20260809 07:33:31.806671 2523 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.122s user 0.001s sys 0.000s
I20260809 07:33:31.874121 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:31.876422 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 51330.5us
I20260809 07:33:31.876876 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.877462 30898 raft_consensus.cc:2379] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:31.876367 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.878880 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.879068 2426 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:31.879585 2426 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.880535 2426 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.874038 2251 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.883198 31177 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:31.882659 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.884989 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.885723 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.886780 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.887631 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.888204 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.887842 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.887602 30899 raft_consensus.cc:2379] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:31.888419 30906 raft_consensus.cc:2454] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:31.889421 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.883576 2251 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.891671 31055 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260809 07:33:31.892549 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:31.893018 31049 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:31.893865 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:31.878080 2430 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s user 0.004s sys 0.041s
W20260809 07:33:31.917354 2430 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s user 0.004s sys 0.042s
I20260809 07:33:31.918040 2430 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:31.830794 2447 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260809 07:33:31.830451 2326 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:31.893116 31051 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:31.928494 2243 consensus_queue.cc:237] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.929180 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:31.929337 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 55610.9us
I20260809 07:33:31.929812 31058 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:31.931288 2427 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:31.930989 2326 raft_consensus.cc:686] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:31.923833 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.924094 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.948699 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.871069 2459 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.920706 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.950480 30898 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:31.925171 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:31.747172 2517 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.197s
I20260809 07:33:31.925352 2524 consensus_queue.cc:1046] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:31.958886 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.925746 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.783193 2485 consensus_queue.cc:1046] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260809 07:33:31.926698 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:31.961760 31054 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:31.927634 31055 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:31.928054 2447 raft_consensus.cc:3032] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:31.921895 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.932526 31006 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:31.923207 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:31.925693 2430 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.875996 2527 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:31.884208 2283 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Leader election won for term 11
I20260809 07:33:31.942866 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.874142 2452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36476 (request call id 59) took 1971 ms. Trace:
I20260809 07:33:31.951795 31174 raft_consensus.cc:2379] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:31.928380 2498 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:31.932269 2458 consensus_meta.cc:287] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.733s user 0.010s sys 0.001s
I20260809 07:33:31.966032 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.968406 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.966411 31048 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:31.970131 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:31.972468 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.918984 2522 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:31.973888 30906 raft_consensus.cc:2379] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28b9965e21374d26b0ac2014c9bafdff in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:31.801450 2483 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.979878 30909 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:31.986018 31055 raft_consensus.cc:1261] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 07:33:31.965783 2528 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:31.988121 2498 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.991377 2527 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.993115 1517 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:31.979259 2446 raft_consensus.cc:2781] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:31.786149 2482 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.965871 2447 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.003239 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:31.996405 30898 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:32.005169 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.008322 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:32.008471 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.009094 30896 raft_consensus.cc:2362] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7. Re-sending same reply.
I20260809 07:33:32.010070 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.011642 30906 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:31.986651 2283 raft_consensus.cc:686] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:31.999788 2522 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.017549 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:31.986366 1856 consensus_queue.cc:1046] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260809 07:33:32.019222 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.019467 30900 raft_consensus.cc:2354] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:32.015285 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.022249 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:32.061510 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 336463us
I20260809 07:33:32.008133 2446 raft_consensus.cc:686] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:31.988981 2458 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.075302 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.082015 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.023801 2482 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.082680 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:32.087489 2522 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.089522 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:32.023336 2427 raft_consensus.cc:3032] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:32.100471 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.101107 30906 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:32.111408 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.111775 31169 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6. Ops: []
W20260809 07:33:32.114169 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 113359us
W20260809 07:33:32.115414 31005 leader_election.cc:334] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.513s (SENT)
I20260809 07:33:32.112093 30898 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:32.119478 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.121594 30891 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:32.123898 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.012903 2459 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.091504 2417 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.099992 2427 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.000597 2528 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.127413 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.138908 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.139586 30898 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:32.118451 2382 raft_consensus.cc:3032] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:32.018702 2326 consensus_queue.cc:237] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:32.142459 31005 leader_election.cc:334] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.518s (SENT)
W20260809 07:33:32.145426 1621 consensus_meta.cc:287] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.523s user 0.005s sys 0.004s
I20260809 07:33:32.148298 2382 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.140710 2353 consensus_queue.cc:1046] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260809 07:33:32.192440 2382 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.193639 2382 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.194118 31165 raft_consensus.cc:2396] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 07:33:32.193559 2353 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:32.195210 2353 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.195436 2515 raft_consensus.cc:3032] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:32.195842 2515 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.196048 2353 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.196748 2515 consensus_queue.cc:260] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.197371 2353 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.198681 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.199285 30900 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
W20260809 07:33:32.205060 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 62627.5us
I20260809 07:33:32.206259 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.207140 30900 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:32.210502 2458 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.212217 2528 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.127386 2283 consensus_queue.cc:237] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.219540 2427 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:32.146570 2498 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.146657 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:32.107306 2517 consensus_queue.cc:1046] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260809 07:33:32.222002 2517 raft_consensus.cc:3032] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:32.131745 31120 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:32.198057 2515 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:32.181556 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:32.181878 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 249420us
I20260809 07:33:32.179930 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:32.190363 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 226502us
W20260809 07:33:32.231670 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 100011us
I20260809 07:33:32.116652 1517 consensus_queue.cc:1046] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260809 07:33:32.089169 2447 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:31.978360 2452 rpcz_store.cc:276] 0809 07:33:29.858138 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:29.858277 (+ 139us) service_pool.cc:227] Handling call
0809 07:33:29.932388 (+ 74111us) log.cc:842] Serialized 494 byte log entry
0809 07:33:31.823668 (+1891280us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":82297,"raft.run_cpu_time_us":1194,"raft.run_wall_time_us":18926,"thread_start_us":228270,"threads_started":2,"wal-append.queue_time_us":162400,"wal-append.run_cpu_time_us":2003,"wal-append.run_wall_time_us":1031333}
W20260809 07:33:32.223592 31005 leader_election.cc:334] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.620s (SENT)
I20260809 07:33:32.251212 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.232542 30898 raft_consensus.cc:2379] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:32.251987 31005 leader_election.cc:334] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.742s (SENT)
I20260809 07:33:32.247113 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:32.252374 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.256376 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.257050 31174 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:32.258723 31177 raft_consensus.cc:1226] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260809 07:33:32.222409 2517 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.222887 2430 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.257154 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.270068 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.270491 31182 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
W20260809 07:33:32.262176 31005 leader_election.cc:334] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.796s (SENT)
I20260809 07:33:32.271210 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.266748 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.273576 31175 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:32.275256 2488 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:32.276592 2553 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:32.280678 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:32.252923 2353 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:32.295954 2353 raft_consensus.cc:686] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.297214 2504 raft_consensus.cc:3032] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:32.296890 2353 consensus_queue.cc:237] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.297609 2504 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.298067 2461 consensus_queue.cc:1046] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:32.298489 2504 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.297060 30853 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.299697 2504 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.300462 30853 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.302767 2500 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.303262 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.285359 2541 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 07:33:32.287204 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:32.302847 31182 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:32.281443 2535 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:32.292924 2483 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.294651 2548 consensus_queue.cc:1046] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:32.183396 2446 consensus_queue.cc:237] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.257622 2382 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:32.298944 2461 raft_consensus.cc:3032] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:32.322798 2461 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.323128 31045 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 4. Current term is 5. Ops: []
I20260809 07:33:32.323649 2461 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.324846 2461 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.304306 2447 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:32.327069 2500 raft_consensus.cc:3032] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:32.327494 2500 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.328336 2500 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.302932 2426 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:32.284909 31005 leader_election.cc:334] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.822s (SENT)
I20260809 07:33:32.284255 30724 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 335fccb907184174b4a6428ce262e7dc with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 07:33:32.331122 31005 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:32.331343 30724 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47217.8us
W20260809 07:33:32.332052 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 47174us
W20260809 07:33:32.332621 31005 leader_election.cc:334] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.891s (SENT)
W20260809 07:33:32.333365 31005 leader_election.cc:334] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.906s (SENT)
I20260809 07:33:32.333683 31005 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.299647 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.308704 31170 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:32.308907 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.290078 30899 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:32.314430 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.316834 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.283215 2430 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:32.320469 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.320981 2452 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.337459 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.342170 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.345228 31174 raft_consensus.cc:2379] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:32.342427 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.342720 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.347208 31006 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.350867 31175 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:32.359072 30853 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.360709 30853 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.361656 30909 raft_consensus.cc:2454] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:32.352106 31172 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260809 07:33:32.363315 31005 leader_election.cc:334] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.908s (SENT)
I20260809 07:33:32.364235 31005 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.367501 30853 leader_election.cc:398] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:32.369732 30853 leader_election.cc:398] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8.
I20260809 07:33:32.370035 30853 leader_election.cc:401] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260809 07:33:32.341372 2535 raft_consensus.cc:686] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.343693 2243 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.344223 2465 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.351645 2426 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:32.381349 31005 leader_election.cc:334] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.931s (SENT)
I20260809 07:33:32.351289 2527 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.356067 2541 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:32.300482 2517 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.385352 30853 leader_election.cc:398] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
I20260809 07:33:32.392598 30853 leader_election.cc:401] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260809 07:33:32.388701 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.370540 2251 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260809 07:33:32.390604 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 121050us
I20260809 07:33:32.371479 2382 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260809 07:33:32.356451 2514 consensus_meta.cc:287] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.694s user 0.014s sys 0.007s
I20260809 07:33:32.379125 2415 raft_consensus.cc:3037] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.347028 2430 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.379354 2524 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.395110 30852 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.421331 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.395895 1621 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:33:32.389405 1856 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.394558 2426 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 5. Current term is 6.
W20260809 07:33:32.421708 31005 leader_election.cc:334] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.944s (SENT)
I20260809 07:33:32.402406 2529 consensus_queue.cc:1046] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260809 07:33:32.392067 2541 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:32.411527 2493 raft_consensus.cc:3032] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:32.378495 2522 consensus_queue.cc:1046] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260809 07:33:32.418622 31164 raft_consensus.cc:1261] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260809 07:33:32.420293 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 54825.1us
I20260809 07:33:32.397452 2527 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.367441 2452 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.397972 2382 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:32.424681 2539 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260809 07:33:32.425529 2417 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:32.399611 30853 leader_election.cc:398] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:32.432659 31181 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:32.433305 31005 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.447934 30852 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a
W20260809 07:33:32.469645 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 70339.7us
I20260809 07:33:32.470464 30853 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:32.471810 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 50423.5us
W20260809 07:33:32.452229 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 135537us
I20260809 07:33:32.420581 2517 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.431115 2458 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:32.393064 2465 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260809 07:33:32.461151 2430 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.461844 32198 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:32.452723 2493 raft_consensus.cc:729] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.484844 2458 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:32.492785 2493 consensus_queue.cc:260] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.493983 2493 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.498852 2482 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 LEADER]: Stepping down as leader of term 10
I20260809 07:33:32.499310 2482 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.492830 2458 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:32.500175 2482 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.501508 2482 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:32.452492 2541 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:32.472496 31005 leader_election.cc:334] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): Timed out: RequestConsensusVote RPC to 127.28.87.65:33749 timed out after 1.945s (SENT)
I20260809 07:33:32.504446 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.506067 31045 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66"
dest_uuid: "a720817fc113446a8ed55e61523b280a"
mode: GRACEFUL
new_leader_uuid: "12107e38eb6d424abc4f45f16a9a1560"
from {username='slave'} at 127.0.0.1:38242
I20260809 07:33:32.466230 2326 raft_consensus.cc:3032] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:32.507134 2326 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.507992 2326 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.509196 2326 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.470134 2417 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:32.456714 2535 consensus_queue.cc:237] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.510514 2417 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:32.486552 2504 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.487160 30853 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.469317 2382 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.488806 2457 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:32.483745 2529 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.488492 30852 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.490350 2561 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
W20260809 07:33:32.491847 2552 log.cc:924] Time spent T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260809 07:33:32.514495 30853 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.488428 2485 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.514765 31045 raft_consensus.cc:595] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Received request to transfer leadership to TS 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:32.522717 2427 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.531387 2527 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.539443 2452 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.541112 2461 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:32.542374 30853 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.525312 1517 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260809 07:33:32.544476 2415 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.544759 2458 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.532130 31045 raft_consensus.cc:603] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:32.547479 30911 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:32.535158 2570 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260809 07:33:32.537400 31182 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 07:33:32.538005 2459 raft_consensus.cc:3032] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 07:33:32.519654 2539 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.533805 2504 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:32.549088 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:32.547035 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.554191 2459 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.558905 30853 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.560976 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.562461 31169 raft_consensus.cc:2379] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:32.568297 30853 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.553040 2243 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.552899 2461 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.611243 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.561723 2504 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:32.541034 2541 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.563362 2458 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.571110 30853 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.572198 1517 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:32.569360 2485 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.554265 2426 consensus_queue.cc:1046] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260809 07:33:32.621858 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.624315 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
W20260809 07:33:32.626085 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 55047.5us
I20260809 07:33:32.583218 2452 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.628515 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.631778 2426 raft_consensus.cc:3032] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:32.632138 2426 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.632905 2426 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.627832 2548 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.622722 2504 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.636641 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.638851 2565 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:32.639521 2565 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.643739 31046 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:32.644732 31048 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:32.646063 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:32.646500 31048 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:32.586576 2415 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260809 07:33:32.650899 31057 raft_consensus.cc:1261] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260809 07:33:32.634382 2457 consensus_queue.cc:1159] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a720817fc113446a8ed55e61523b280a" responder_term: 5 status { last_received { term: 5 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 5 index: 3 } }
I20260809 07:33:32.653210 2353 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:32.655494 2353 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.655951 1517 raft_consensus.cc:686] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.657166 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.656894 1517 consensus_queue.cc:237] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.658485 2517 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:32.659600 31051 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:32.660490 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.663668 2524 raft_consensus.cc:3032] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:32.664032 2524 raft_consensus.cc:729] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.664786 2524 consensus_queue.cc:260] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:32.665916 2458 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.666549 2458 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.658231 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.676604 31046 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:32.634122 2426 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:32.625849 2243 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.629453 2527 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260809 07:33:32.681833 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 53387.6us
I20260809 07:33:32.666015 2524 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:32.691787 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 62274.5us
I20260809 07:33:32.693521 31048 raft_consensus.cc:2454] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:32.694239 31005 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.695472 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.696718 31182 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:32.698180 30853 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.701164 31048 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:32.703290 2461 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:32.703799 31120 leader_election.cc:398] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8.
I20260809 07:33:32.704128 31120 leader_election.cc:401] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:32.704680 2353 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:32.704785 2553 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8.
I20260809 07:33:32.705384 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.705050 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.694505 2415 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.710507 31182 raft_consensus.cc:1261] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:32.711477 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.713428 31169 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:32.714753 2570 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.730057 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.715481 2382 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.716679 2417 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.695153 2579 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.715231 2504 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.624230 2459 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.708640 2465 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.721627 2485 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:32.716723 2427 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.723245 2541 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.723229 31005 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.724638 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.775555 31181 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:32.725816 31175 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:32.727388 31172 raft_consensus.cc:1261] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260809 07:33:32.727959 2461 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:32.778024 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 54875.5us
I20260809 07:33:32.778380 2569 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:32.774132 2565 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260809 07:33:32.764930 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.733358 31170 raft_consensus.cc:1226] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
W20260809 07:33:32.718459 2571 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260809 07:33:32.706450 2243 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:32.774407 31055 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 07:33:32.776259 31172 pending_rounds.cc:76] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 4
I20260809 07:33:32.801088 31172 pending_rounds.cc:97] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260809 07:33:32.801386 31172 raft_consensus.cc:2866] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:32.769240 2446 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:32.779304 31005 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.715101 2527 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:32.779573 2382 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.780138 2427 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.781898 2504 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.769316 2452 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.783778 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.783963 2417 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.777896 2461 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.815729 31005 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:32.816118 2461 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.706132 2539 consensus_queue.cc:1035] T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260809 07:33:32.816910 31005 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.777107 2457 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
W20260809 07:33:32.786882 2578 log.cc:924] Time spent T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.162s user 0.002s sys 0.000s
I20260809 07:33:32.787583 2458 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.788810 1856 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8.
I20260809 07:33:32.797865 31181 raft_consensus.cc:2454] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
W20260809 07:33:32.813522 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 102076us
I20260809 07:33:32.813585 31175 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.819636 2541 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:32.820114 2541 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.812693 2565 consensus_queue.cc:1225] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:32.818427 2569 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 LEADER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 07:33:32.821029 2569 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:32.821475 2590 log.cc:924] Time spent T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.050s user 0.000s sys 0.004s
I20260809 07:33:32.818846 2452 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.823555 2452 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.823705 31006 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.824483 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.825242 2452 raft_consensus.cc:2781] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:32.825814 2452 raft_consensus.cc:686] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.770706 2283 consensus_queue.cc:1035] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260809 07:33:32.826691 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.826975 2452 consensus_queue.cc:237] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.828799 31175 raft_consensus.cc:2454] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:32.829726 2446 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.830646 31120 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.831403 2482 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.831813 2446 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.832923 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.833617 31005 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.828928 2500 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.836416 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.836512 31006 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.837603 31005 leader_election.cc:398] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:32.837973 31005 leader_election.cc:401] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:32.838454 30745 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: config changed from index 6 to 7, VOTER 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) evicted. New cstate: current_term: 5 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:32.839083 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.840860 2528 raft_consensus.cc:2781] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:32.845371 2570 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.846228 31181 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:32.847793 2382 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:32.803272 2515 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:32.841969 31005 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:32.852605 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 84413.4us
I20260809 07:33:32.855464 31181 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:32.837028 2379 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.845445 2528 raft_consensus.cc:686] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:32.857734 31005 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.841216 2446 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:32.849483 2527 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.835248 2243 raft_consensus.cc:686] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.855160 2382 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.876840 30853 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.856451 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.880434 31182 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:32.878260 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.879112 31005 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.879290 31165 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 11 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260809 07:33:32.876399 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:32.881832 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.877703 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:32.884387 31165 pending_rounds.cc:76] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 6
I20260809 07:33:32.884637 31165 pending_rounds.cc:97] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260809 07:33:32.884859 31165 raft_consensus.cc:2866] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:32.887130 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.888135 2482 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260809 07:33:32.886118 2515 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:32.890031 2515 raft_consensus.cc:686] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:32.890992 2515 consensus_queue.cc:237] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.892025 2528 consensus_queue.cc:237] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.866250 2483 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.874472 2447 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.820988 2541 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.887042 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:32.885846 2566 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.286s user 0.005s sys 0.000s
I20260809 07:33:32.899577 31174 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 07:33:32.888381 2535 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.855212 1517 consensus_queue.cc:1035] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260809 07:33:32.866184 2326 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260809 07:33:32.902380 30852 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.907330 2498 consensus_queue.cc:1046] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 07:33:32.903604 2586 log.cc:924] Time spent T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.053s user 0.000s sys 0.004s
I20260809 07:33:32.908965 2465 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:32.900354 2459 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 4 FOLLOWER]: Advancing to term 5
I20260809 07:33:32.911137 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:32.929752 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.931625 31058 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:32.931954 31005 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:32.922747 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:32.932909 31059 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:32.923035 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:32.908224 2569 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.899848 2382 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.899875 2415 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 07:33:32.925783 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.943519 31046 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:32.939564 31059 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:32.944845 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:32.938212 31048 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:32.939863 31005 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.913811 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.947410 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.948371 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.899639 2283 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260809 07:33:32.945529 31051 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:32.919932 2504 raft_consensus.cc:2781] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:32.953370 31005 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:32.955371 30852 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.958715 31048 raft_consensus.cc:2454] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:32.944494 2524 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:32.945050 2446 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:32.910993 2483 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.973872 31005 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:32.975055 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.949234 2522 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.956180 2569 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:32.978221 30852 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.982663 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.951637 1517 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:32.926654 2458 consensus_queue.cc:1046] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260809 07:33:32.951711 2535 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:32.955156 2457 raft_consensus.cc:3032] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:32.986032 30852 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.976892 2524 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.961261 2243 consensus_queue.cc:237] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:32.991000 2593 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.973595 2427 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.979698 2447 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:32.959113 2575 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:33.013950 2595 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
W20260809 07:33:33.015466 2592 log.cc:924] Time spent T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.137s user 0.000s sys 0.000s
I20260809 07:33:33.015995 2575 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:33.016438 2575 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:32.998611 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:32.979592 2446 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:32.982525 2415 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:32.999782 2353 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260809 07:33:32.978880 2504 raft_consensus.cc:686] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:33.003925 2483 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:32.981259 2570 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:33.006788 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:32.996935 2565 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:32.998481 2522 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:33.012889 2447 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.024724 30852 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:33.008061 2524 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:33.031442 2522 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.011221 2498 raft_consensus.cc:3032] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:33.038429 2522 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:33.038872 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 130492us
I20260809 07:33:33.040400 30853 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:33.040802 2427 consensus_queue.cc:1035] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:33.041100 2283 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:33.042202 2427 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:33.038086 30852 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:32.997752 2457 raft_consensus.cc:729] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:32.999007 2515 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:33.022235 2446 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:33.045349 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:33.045656 31048 raft_consensus.cc:1261] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260809 07:33:33.024075 2415 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.025993 2251 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:33.028003 2605 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:33.054739 31051 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 07:33:33.021839 2593 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.057184 1517 consensus_queue.cc:1035] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:33.050154 31045 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:33.031570 2482 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 07:33:33.059672 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 61153.4us
I20260809 07:33:33.046072 2498 raft_consensus.cc:729] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:33.044467 2447 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:33.078358 2483 consensus_queue.cc:1035] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:33.079011 2593 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:33.066572 2524 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.080641 2251 raft_consensus.cc:686] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:33.081404 2524 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:33.081518 2251 consensus_queue.cc:237] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.082892 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:33.083029 2417 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:33:33.058104 2570 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260809 07:33:33.084231 2309 raft_consensus.cc:3032] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
W20260809 07:33:33.084532 2596 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.193s user 0.000s sys 0.002s
I20260809 07:33:33.049896 2430 raft_consensus.cc:2781] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:33.086261 2430 raft_consensus.cc:686] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:33.087154 2430 consensus_queue.cc:237] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.090719 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.090807 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.060459 2415 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:33.030601 2565 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.069881 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.038383 2539 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:33.072086 31045 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260809 07:33:33.074677 30744 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40582 (request call id 150) took 8028 ms. Trace:
I20260809 07:33:33.114120 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:33.125998 30909 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:33.135241 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:33.114444 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.137120 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:33.138218 31165 raft_consensus.cc:1226] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260809 07:33:33.138476 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:33.115973 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:33.139941 31046 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:33.140659 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.138839 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:33.141954 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:33.142709 31051 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:33.139851 31177 raft_consensus.cc:1261] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 07:33:33.043269 2504 consensus_queue.cc:237] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.145808 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.146677 30898 raft_consensus.cc:2379] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:33.147672 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:33.148307 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:33.149374 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 79577.1us
I20260809 07:33:33.141331 30744 rpcz_store.cc:276] 0809 07:33:25.046463 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:25.070123 (+ 23660us) service_pool.cc:227] Handling call
0809 07:33:33.074610 (+8004487us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:31.979300 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:32.075868 (+ 96568us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:32.079320 (+ 3452us) write_op.cc:435] Acquired schema lock
0809 07:33:32.079339 (+ 19us) tablet.cc:660] Decoding operations
0809 07:33:32.144975 (+ 65636us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:32.145035 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:32.145060 (+ 25us) tablet.cc:683] Acquiring locks for 20 operations
0809 07:33:32.146381 (+ 1321us) tablet.cc:699] Row locks acquired
0809 07:33:32.146402 (+ 21us) write_op.cc:260] PREPARE: finished
0809 07:33:32.146538 (+ 136us) write_op.cc:270] Start()
0809 07:33:32.146646 (+ 108us) write_op.cc:276] Timestamp: P: 1786260812146493 usec, L: 0
0809 07:33:32.146660 (+ 14us) op_driver.cc:331] REPLICATION: starting
0809 07:33:32.179553 (+ 32893us) log.cc:842] Serialized 7849 byte log entry
0809 07:33:32.380434 (+200881us) op_driver.cc:444] REPLICATION: finished
0809 07:33:32.718853 (+338419us) write_op.cc:301] APPLY: starting
0809 07:33:32.718959 (+ 106us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:32.719321 (+ 362us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:32.719429 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:32.721207 (+ 1778us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:32.721531 (+ 324us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0809 07:33:32.721574 (+ 43us) write_op.cc:312] APPLY: finished
0809 07:33:32.721919 (+ 345us) log.cc:842] Serialized 138 byte log entry
0809 07:33:32.769406 (+ 47487us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:32.774244 (+ 4838us) write_op.cc:454] Released schema lock
0809 07:33:32.778971 (+ 4727us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:32.779266 (+ 295us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:32.779294 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75361,"reported_tablets":58,"child_traces":[["op",{"apply.queue_time_us":325954,"apply.run_cpu_time_us":4458,"apply.run_wall_time_us":60903,"num_ops":20,"prepare.queue_time_us":361185,"prepare.run_cpu_time_us":24823,"prepare.run_wall_time_us":327753,"raft.queue_time_us":63014,"raft.run_cpu_time_us":55987,"raft.run_wall_time_us":283250,"replication_time_us":230514,"thread_start_us":449077,"threads_started":4,"wal-append.queue_time_us":93731}]]}
I20260809 07:33:33.149446 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:33.155380 31057 raft_consensus.cc:2396] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260809 07:33:33.092592 2309 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:33.156582 30890 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 6: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:33.156450 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:33.130759 2565 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:33.153812 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.162849 30904 raft_consensus.cc:1226] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:33.167898 2548 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:33.089761 2595 consensus_queue.cc:1046] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260809 07:33:33.114437 2458 consensus_queue.cc:1035] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260809 07:33:33.128819 2539 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.175343 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.169762 2575 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.179394 31046 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:33.184197 31055 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:33.156879 2452 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:33.218349 30898 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260809 07:33:33.220079 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 72038.5us
I20260809 07:33:33.223297 31046 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260809 07:33:33.225950 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 50675.8us
I20260809 07:33:33.226720 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.228721 30891 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:33.230703 31120 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:33.232551 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 97837.8us
I20260809 07:33:33.233067 31058 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:33.233702 31046 raft_consensus.cc:1226] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260809 07:33:33.185393 2539 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:33.215032 2541 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:33.233335 30852 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:33.268965 2605 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:33.151154 2457 consensus_queue.cc:260] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.279945 2457 raft_consensus.cc:3037] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:33.231868 2251 raft_consensus.cc:3032] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:33.228304 2528 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260809 07:33:33.289642 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 64762us
I20260809 07:33:33.292690 31058 raft_consensus.cc:1261] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:33.295097 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.310622 31182 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 07:33:33.296972 2427 consensus_queue.cc:1035] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 07:33:33.328845 31058 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:33.329216 31169 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:33.329871 31169 raft_consensus.cc:595] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:33.330215 31169 raft_consensus.cc:603] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260809 07:33:33.331032 2613 log.cc:924] Time spent T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260809 07:33:33.335399 2498 consensus_queue.cc:260] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.336484 2498 raft_consensus.cc:3037] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:33.308559 2515 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 07:33:33.308377 2605 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.286751 2251 raft_consensus.cc:729] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:33.340504 2605 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.295143 2561 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:33.302416 2458 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:33.240279 2309 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.276335 2522 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.365522 30906 raft_consensus.cc:1226] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260809 07:33:33.367609 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 66855us
I20260809 07:33:33.370591 31182 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:33.366039 2561 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:33.327297 2415 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.380285 31177 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
W20260809 07:33:33.384953 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 151692us
I20260809 07:33:33.410179 31058 raft_consensus.cc:1226] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:33.410705 31182 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:33.363004 2458 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.415431 30891 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:33.420077 2382 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:33.420334 32141 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.420687 2415 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:33.417598 2524 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260809 07:33:33.366392 1517 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260809 07:33:33.341243 2251 consensus_queue.cc:260] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.415537 2458 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:33.281683 2417 consensus_queue.cc:1159] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12107e38eb6d424abc4f45f16a9a1560" responder_term: 9 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]" } } last_received_current_leader { term: 0 index: 0 } }
W20260809 07:33:33.469900 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 49649us
I20260809 07:33:33.389199 2535 consensus_queue.cc:1046] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260809 07:33:33.479621 2522 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:33.480038 2630 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.086s user 0.001s sys 0.000s
W20260809 07:33:33.481062 2634 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260809 07:33:33.422016 2447 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.461954 2575 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:33.514083 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:33.513808 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 125742us
I20260809 07:33:33.518757 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:33.542814 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 134296us
I20260809 07:33:33.543673 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:33.544495 30911 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:33.482695 2515 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:33.546211 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:33.596151 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:33.596562 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 49719.8us
I20260809 07:33:33.596457 31120 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:33.600927 30911 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
W20260809 07:33:33.600782 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 50634.4us
I20260809 07:33:33.543691 2283 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:33.600529 31005 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:33.548838 2575 raft_consensus.cc:3032] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:33.620993 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.634117 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:33.635437 31045 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:33.661731 31045 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:33.661301 31120 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:33.688725 31059 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:33.632068 2251 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:33.635840 2605 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:33.599665 2483 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260809 07:33:33.595363 2515 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:33.750046 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 149592us
I20260809 07:33:33.750856 31054 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:33.752508 30911 raft_consensus.cc:2454] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
W20260809 07:33:33.703123 2379 consensus_meta.cc:287] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.831s user 0.011s sys 0.000s
I20260809 07:33:33.660187 2575 raft_consensus.cc:729] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:33.757115 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 210958us
I20260809 07:33:33.513598 2561 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.761287 2465 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:33.755440 2595 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:33.792714 2605 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:33.834105 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.846009 30911 raft_consensus.cc:2362] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10. Re-sending same reply.
W20260809 07:33:33.832531 2639 log.cc:924] Time spent T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
I20260809 07:33:33.838160 2379 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:33.856230 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.832146 2535 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:33.837813 2465 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:33.865366 31055 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:33.842613 2447 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:33.883390 2515 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:33.874722 2465 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:33.845108 2493 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260809 07:33:33.889638 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:33.890195 2643 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:33.890915 30909 raft_consensus.cc:1226] T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
W20260809 07:33:33.858533 2539 consensus_meta.cc:287] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.583s user 0.005s sys 0.005s
I20260809 07:33:33.842173 2651 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:33.846468 2595 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.892366 2504 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:33.886250 2565 consensus_meta.cc:287] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.717s user 0.007s sys 0.000s
W20260809 07:33:33.898653 2633 log.cc:924] Time spent T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.400s user 0.000s sys 0.001s
I20260809 07:33:33.853295 2605 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:33.897571 2498 consensus_meta.cc:287] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.561s user 0.009s sys 0.001s
I20260809 07:33:33.917956 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:33.943390 30900 raft_consensus.cc:1226] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260809 07:33:33.947149 2632 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.233s user 0.001s sys 0.000s
I20260809 07:33:33.920969 2461 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:33.912074 2657 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
W20260809 07:33:33.950567 2641 log.cc:924] Time spent T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.124s user 0.000s sys 0.000s
W20260809 07:33:33.951040 2646 log.cc:924] Time spent T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.287s user 0.000s sys 0.001s
I20260809 07:33:33.951344 2626 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:33.951798 2647 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.138s user 0.000s sys 0.000s
W20260809 07:33:33.952257 2644 log.cc:924] Time spent T 1a18497352404d2491d49a9241fe61bf P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
W20260809 07:33:33.952585 2653 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
W20260809 07:33:33.953760 2652 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.154s user 0.001s sys 0.000s
I20260809 07:33:33.920501 2595 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:33.935005 2565 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.945742 2515 raft_consensus.cc:3032] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:33.952360 31177 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260809 07:33:33.956880 2541 consensus_meta.cc:287] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.680s user 0.004s sys 0.010s
I20260809 07:33:33.957566 2665 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.942836 2504 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
W20260809 07:33:33.961644 2616 log.cc:924] Time spent T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.731s user 0.012s sys 0.016s
I20260809 07:33:33.963778 2458 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.965500 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 26) took 2450 ms. Trace:
W20260809 07:33:33.968019 2629 log.cc:924] Time spent T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.372s user 0.000s sys 0.000s
I20260809 07:33:33.970351 2465 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:33.971194 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet 335fccb907184174b4a6428ce262e7dc with delete_type TABLET_DATA_TOMBSTONED (TS 2c9de56c3fa3487c9dac990675c34be5 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43208
I20260809 07:33:33.956019 2677 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:33.972196 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:34.080593 30906 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:33.978811 2640 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.504s user 0.000s sys 0.003s
I20260809 07:33:33.980996 2660 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:33.981097 31182 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260809 07:33:33.983021 2459 consensus_meta.cc:287] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.036s user 0.010s sys 0.001s
W20260809 07:33:33.984863 2446 consensus_meta.cc:287] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.929s user 0.010s sys 0.000s
I20260809 07:33:34.109053 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1478) took 1178 ms. Trace:
I20260809 07:33:33.991458 30909 raft_consensus.cc:2362] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7. Re-sending same reply.
I20260809 07:33:33.921006 2659 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:33.898238 2575 consensus_queue.cc:260] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.074775 30890 raft_consensus.cc:1226] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:34.072223 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.013595 2657 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:34.066531 30745 rpcz_store.cc:276] 0809 07:33:31.518538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:31.523495 (+ 4957us) service_pool.cc:227] Handling call
0809 07:33:33.965413 (+2441918us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:33.074864 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:33.075271 (+ 407us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:33.075297 (+ 26us) write_op.cc:435] Acquired schema lock
0809 07:33:33.075314 (+ 17us) tablet.cc:660] Decoding operations
0809 07:33:33.075593 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:33.075651 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:33.075667 (+ 16us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:33:33.075815 (+ 148us) tablet.cc:699] Row locks acquired
0809 07:33:33.075826 (+ 11us) write_op.cc:260] PREPARE: finished
0809 07:33:33.075934 (+ 108us) write_op.cc:270] Start()
0809 07:33:33.076010 (+ 76us) write_op.cc:276] Timestamp: P: 1786260813075900 usec, L: 0
0809 07:33:33.076022 (+ 12us) op_driver.cc:331] REPLICATION: starting
0809 07:33:33.076636 (+ 614us) log.cc:842] Serialized 528 byte log entry
0809 07:33:33.143420 (+ 66784us) op_driver.cc:444] REPLICATION: finished
0809 07:33:33.164687 (+ 21267us) write_op.cc:301] APPLY: starting
0809 07:33:33.164767 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:33.164866 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:33.164963 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:33.165146 (+ 183us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:33.165302 (+ 156us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:33:33.165339 (+ 37us) write_op.cc:312] APPLY: finished
0809 07:33:33.165506 (+ 167us) log.cc:842] Serialized 22 byte log entry
0809 07:33:33.165650 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:33.165730 (+ 80us) write_op.cc:454] Released schema lock
0809 07:33:33.165764 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:33.166030 (+ 266us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:33.166054 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":45279,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":4204,"apply.run_cpu_time_us":1567,"apply.run_wall_time_us":1566,"num_ops":1,"prepare.queue_time_us":203009,"prepare.run_cpu_time_us":2163,"prepare.run_wall_time_us":42239,"replication_time_us":55729,"thread_start_us":202698,"threads_started":1}]]}
I20260809 07:33:34.022887 2515 raft_consensus.cc:729] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:33.917104 2539 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.063534 2541 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:34.013957 2461 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.109467 31049 rpcz_store.cc:276] 0809 07:33:32.931064 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:32.931210 (+ 146us) service_pool.cc:227] Handling call
0809 07:33:32.951969 (+ 20759us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:32.975494 (+ 23525us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:32.975524 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:32.975958 (+ 434us) log.cc:842] Serialized 26 byte log entry
0809 07:33:33.484370 (+508412us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:33.484431 (+ 61us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:33.484542 (+ 111us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:34.108863 (+624321us) raft_consensus.cc:1671] finished
0809 07:33:34.108881 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:34.108997 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6012,"raft.queue_time_us":5604,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4956,"thread_start_us":452172,"threads_started":2,"wal-append.queue_time_us":534518}
I20260809 07:33:34.143350 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:34.016197 2626 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.147599 2659 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.172960 30911 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:34.177246 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1294) took 1040 ms. Trace:
I20260809 07:33:34.109942 2498 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:34.177892 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.191924 31170 rpcz_store.cc:276] 0809 07:33:33.136756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.137098 (+ 342us) service_pool.cc:227] Handling call
0809 07:33:33.137390 (+ 292us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:33.137715 (+ 325us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:33.137742 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:33.138168 (+ 426us) log.cc:842] Serialized 26 byte log entry
0809 07:33:33.299411 (+161243us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:33.299446 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:33.299512 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:33.967358 (+667846us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:34.156148 (+188790us) raft_consensus.cc:1671] finished
0809 07:33:34.156171 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:34.177172 (+ 21001us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4669,"raft.queue_time_us":1365,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":9583,"spinlock_wait_cycles":38899584,"thread_start_us":128798,"threads_started":1,"wal-append.queue_time_us":312717}
I20260809 07:33:34.199093 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1279) took 1299 ms. Trace:
I20260809 07:33:34.200453 30899 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
W20260809 07:33:34.173009 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.001s sys 0.013s
W20260809 07:33:34.201821 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.538s user 0.001s sys 0.013s
W20260809 07:33:34.202837 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 541586us
I20260809 07:33:34.146615 2459 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.200026 31181 rpcz_store.cc:276] 0809 07:33:32.875374 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:32.878256 (+ 2882us) service_pool.cc:227] Handling call
0809 07:33:32.886204 (+ 7948us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:32.886551 (+ 347us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:32.886579 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:32.913845 (+ 27266us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b18003aa380. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f9ef96 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a
0809 07:33:32.914162 (+ 317us) log.cc:842] Serialized 26 byte log entry
0809 07:33:33.056312 (+142150us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:33.056355 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:33.056416 (+ 61us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:33.961217 (+904801us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b2400251d38. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091e29cf4 00007f4091e27dfb 00007f4091eb6559 00007f4091eb61e5 00007f4091eb582d 00007f4094b6446e 00007f4094b7bc0c 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551
0809 07:33:34.174733 (+213516us) raft_consensus.cc:1671] finished
0809 07:33:34.174758 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:34.174896 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":6650,"spinlock_wait_cycles":84246912,"thread_start_us":141301,"threads_started":1,"wal-append.queue_time_us":236428}
I20260809 07:33:34.104522 2535 raft_consensus.cc:686] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:34.157110 2457 consensus_meta.cc:287] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.863s user 0.004s sys 0.007s
I20260809 07:33:34.198487 2651 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:34.111387 2675 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260809 07:33:34.156961 2575 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:34.143297 2446 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.063812 2665 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.104079 2660 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.099221 2677 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:34.204274 2663 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
I20260809 07:33:34.207281 2541 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.290791 31046 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:34.257828 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:34.214699 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 137826us
I20260809 07:33:34.239926 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:34.321167 30896 raft_consensus.cc:2396] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260809 07:33:34.314229 30906 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:34.324509 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.325027 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.325165 31172 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.325660 31170 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
W20260809 07:33:34.237838 2645 log.cc:924] Time spent T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.403s user 0.000s sys 0.002s
W20260809 07:33:34.244836 2685 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.091s user 0.000s sys 0.000s
I20260809 07:33:34.332422 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.334797 30906 raft_consensus.cc:2396] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260809 07:33:34.335320 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1133) took 1021 ms. Trace:
I20260809 07:33:34.333050 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1145) took 1015 ms. Trace:
I20260809 07:33:34.335983 30896 rpcz_store.cc:276] 0809 07:33:33.317538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.216731 (+899193us) service_pool.cc:227] Handling call
0809 07:33:34.332983 (+116252us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.227766 2457 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.255324 2465 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.335861 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.335803 30906 rpcz_store.cc:276] 0809 07:33:33.313795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.332143 (+1018348us) service_pool.cc:227] Handling call
0809 07:33:34.335253 (+ 3110us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.251142 2461 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.390358 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:34.243696 2504 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:34.295477 2690 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.291268 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
W20260809 07:33:34.394904 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 55767us
I20260809 07:33:34.397591 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.398869 30906 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 9: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:34.399289 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1149) took 1080 ms. Trace:
I20260809 07:33:34.399607 30906 rpcz_store.cc:276] 0809 07:33:33.318704 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.397311 (+1078607us) service_pool.cc:227] Handling call
0809 07:33:34.399224 (+ 1913us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.400532 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.203892 2498 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.334416 2605 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.406096 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:34.246568 2561 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.151s
I20260809 07:33:34.213908 2459 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.410109 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.411016 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:34.412268 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.570s user 0.012s sys 0.032s
I20260809 07:33:34.411065 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1142) took 1094 ms. Trace:
I20260809 07:33:34.413967 30906 rpcz_store.cc:276] 0809 07:33:33.316475 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.409868 (+1093393us) service_pool.cc:227] Handling call
0809 07:33:34.411009 (+ 1141us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.395049 2565 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.395987 1856 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.414908 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:34.412935 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.570s user 0.012s sys 0.032s
I20260809 07:33:34.415845 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1136) took 1101 ms. Trace:
I20260809 07:33:34.425793 30896 rpcz_store.cc:276] 0809 07:33:33.314726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.397524 (+1082798us) service_pool.cc:227] Handling call
0809 07:33:34.415768 (+ 18244us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.250514 2657 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.428905 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1333) took 1311 ms. Trace:
I20260809 07:33:34.429608 1856 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.429986 2656 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260809 07:33:34.430763 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1332) took 1313 ms. Trace:
I20260809 07:33:34.403158 2527 raft_consensus.cc:465] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 LEADER]: Not starting pre-election -- already a leader
I20260809 07:33:34.416803 31055 raft_consensus.cc:1203] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Deduplicated request from leader. Original: 4.4->[6.5-6.5] Dedup: 6.5->[]
I20260809 07:33:34.231103 2643 consensus_queue.cc:1035] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.172s
I20260809 07:33:34.417500 2677 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.418004 2561 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260809 07:33:34.418397 2692 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.165s user 0.000s sys 0.000s
I20260809 07:33:34.240468 2680 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.211s
I20260809 07:33:34.419677 2660 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.420928 2539 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.421473 2626 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:34.406262 32198 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.330978 2541 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.394639 1621 consensus_queue.cc:1046] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260809 07:33:34.394512 2457 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:34.423709 2251 consensus_meta.cc:287] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.675s user 0.007s sys 0.003s
I20260809 07:33:34.423489 2493 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 07:33:34.424540 2610 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.095s user 0.000s sys 0.000s
I20260809 07:33:34.247674 2687 consensus_queue.cc:1046] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.155s
I20260809 07:33:34.225752 2515 consensus_queue.cc:260] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.427469 30906 raft_consensus.cc:2396] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260809 07:33:34.397804 2458 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.428478 31049 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:34.429276 30891 rpcz_store.cc:276] 0809 07:33:33.117618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.231401 (+113783us) service_pool.cc:227] Handling call
0809 07:33:33.275900 (+ 44499us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:33.416954 (+141054us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.428629 (+1011675us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54003aded4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.428835 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":320480000}
W20260809 07:33:34.409643 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 86800.8us
I20260809 07:33:34.434144 31055 raft_consensus.cc:1226] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 07:33:34.431128 30898 rpcz_store.cc:276] 0809 07:33:33.117358 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.231399 (+114041us) service_pool.cc:227] Handling call
0809 07:33:33.275660 (+ 44261us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:33.277151 (+ 1491us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:33.277180 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:33.277621 (+ 441us) log.cc:842] Serialized 26 byte log entry
0809 07:33:33.978096 (+700475us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:34.430217 (+452121us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b1800405d60. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4094b8882c 00007f4091fb39dc 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411
0809 07:33:34.430391 (+ 174us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:34.430458 (+ 67us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.430555 (+ 97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:34.430596 (+ 41us) raft_consensus.cc:1671] finished
0809 07:33:34.430606 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:34.430713 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3379,"raft.queue_time_us":79337,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":17532,"spinlock_wait_cycles":14948096,"thread_start_us":62583,"threads_started":1,"wal-append.queue_time_us":126190}
I20260809 07:33:34.436537 30909 raft_consensus.cc:2362] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7. Re-sending same reply.
I20260809 07:33:34.249763 2683 consensus_queue.cc:1046] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.198s
I20260809 07:33:34.473469 30853 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.474082 2683 raft_consensus.cc:3032] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:34.225518 2651 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:34.474351 30853 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.476235 2651 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.477252 30898 raft_consensus.cc:1261] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:34.439404 30896 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:34.478812 30896 raft_consensus.cc:595] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.479156 30896 raft_consensus.cc:603] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:34.454710 2561 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.485129 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:54230 (request call id 15) took 1145 ms. Trace:
I20260809 07:33:34.485294 2541 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.485688 2498 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.486168 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:34.482991 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1336) took 1359 ms. Trace:
I20260809 07:33:34.487480 30898 rpcz_store.cc:276] 0809 07:33:33.118525 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.476586 (+1358061us) service_pool.cc:227] Handling call
0809 07:33:34.476926 (+ 340us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.477615 (+ 689us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.477777 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.488471 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:34.489053 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:34.485502 30896 rpcz_store.cc:276] 0809 07:33:33.334577 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.426436 (+1091859us) service_pool.cc:227] Handling call
0809 07:33:34.479536 (+ 53100us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.491111 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1325) took 1386 ms. Trace:
I20260809 07:33:34.491482 30904 rpcz_store.cc:276] 0809 07:33:33.104358 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.136563 (+ 32205us) service_pool.cc:227] Handling call
0809 07:33:33.136933 (+ 370us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:33.171182 (+ 34249us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.490861 (+1319679us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b54002503d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.491051 (+ 190us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":59015424}
I20260809 07:33:34.250998 2535 consensus_queue.cc:237] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.492736 30904 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:34.493270 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1140) took 1177 ms. Trace:
I20260809 07:33:34.493566 30904 rpcz_store.cc:276] 0809 07:33:33.315970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.492336 (+1176366us) service_pool.cc:227] Handling call
0809 07:33:34.492595 (+ 259us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.493072 (+ 477us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.493223 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.429404 2669 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.496282 2426 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:34.495968 2669 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:34.496695 2426 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:34.496690 2719 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:34.497313 2719 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:34.497915 2719 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.433921 2465 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.450531 31059 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:34.453935 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1346) took 1303 ms. Trace:
I20260809 07:33:34.458655 2714 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.459606 2659 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:34.459940 2602 log.cc:924] Time spent T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.370s user 0.007s sys 0.000s
I20260809 07:33:34.459781 2626 raft_consensus.cc:3032] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:34.402582 2690 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:34.462754 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1330) took 1357 ms. Trace:
I20260809 07:33:34.470242 2687 raft_consensus.cc:3032] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:34.470875 1856 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.471422 2665 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.472663 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1146) took 1154 ms. Trace:
I20260809 07:33:34.491838 30896 raft_consensus.cc:1226] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:34.498461 2426 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.429788 2672 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.499053 2417 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.499526 2457 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.500135 2628 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.500476 2561 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.477960 2683 raft_consensus.cc:729] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:34.503429 2251 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.507079 2498 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.507898 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.590932 31172 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.592034 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.592235 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.592681 31172 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.593505 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.594017 30853 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.511778 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.515082 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.516135 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.521258 30891 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:34.522432 2595 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.498232 2669 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:34.524716 2459 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.530777 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1138) took 1215 ms. Trace:
I20260809 07:33:34.488848 30898 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:34.541900 30904 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260809 07:33:34.542505 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 141822us
I20260809 07:33:34.466382 2446 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.548352 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.440945 31005 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:34.554847 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.561088 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.558501 31051 rpcz_store.cc:276] 0809 07:33:33.150624 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.151612 (+ 988us) service_pool.cc:227] Handling call
0809 07:33:33.179682 (+ 28070us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:33.180020 (+ 338us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:33.180047 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:33.185225 (+ 5178us) log.cc:842] Serialized 26 byte log entry
0809 07:33:33.365565 (+180340us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:33.497862 (+132297us) log.cc:842] Serialized 14 byte log entry
0809 07:33:33.548215 (+ 50353us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:33.548268 (+ 53us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:33.878882 (+330614us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b1800326e00. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:33.881907 (+ 3025us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b1800326e00. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4098da1bcf 00007f4091fd17b6 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d
0809 07:33:33.966435 (+ 84528us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:34.453114 (+486679us) raft_consensus.cc:1671] finished
0809 07:33:34.453134 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:34.453872 (+ 738us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":135945,"prepare.queue_time_us":60263,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":55304,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":32750,"spinlock_wait_cycles":144833792,"thread_start_us":171918,"threads_started":2,"wal-append.queue_time_us":229474}
I20260809 07:33:34.572146 30890 rpcz_store.cc:276] 0809 07:33:33.105539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.176314 (+ 70775us) service_pool.cc:227] Handling call
0809 07:33:33.222852 (+ 46538us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.142724 (+919872us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.462535 (+319811us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b54002508d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.462696 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1958780160}
I20260809 07:33:34.574842 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1135) took 1260 ms. Trace:
I20260809 07:33:34.572659 30906 rpcz_store.cc:276] 0809 07:33:33.317818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.414665 (+1096847us) service_pool.cc:227] Handling call
0809 07:33:34.472601 (+ 57936us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.574299 2687 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:34.553790 2569 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.582906 2659 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.584291 2664 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.584933 2522 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:34.586755 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:34.595945 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.557950 2643 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.510746 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:34.594264 31172 raft_consensus.cc:2379] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:34.600916 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:34.598018 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1337) took 1479 ms. Trace:
I20260809 07:33:34.571943 2626 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:34.589979 30852 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.586460 2665 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.562541 2465 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.616585 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.610225 31182 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:34.617213 31051 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.597667 30911 rpcz_store.cc:276] 0809 07:33:33.315252 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.855016 (+539764us) service_pool.cc:227] Handling call
0809 07:33:34.530592 (+675576us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54001a40d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:34.530712 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":661261184}
I20260809 07:33:34.610278 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.618716 31049 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.619513 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.599793 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1335) took 1481 ms. Trace:
I20260809 07:33:34.620275 31049 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
W20260809 07:33:34.604054 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 163193us
I20260809 07:33:34.605759 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:34.606742 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:34.621752 30906 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.603070 30896 rpcz_store.cc:276] 0809 07:33:33.314417 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.491339 (+1176922us) service_pool.cc:227] Handling call
0809 07:33:34.491657 (+ 318us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.574537 (+ 82880us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.574765 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.621956 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:34.617228 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.623275 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:34.613142 30898 rpcz_store.cc:276] 0809 07:33:33.118777 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.488221 (+1369444us) service_pool.cc:227] Handling call
0809 07:33:34.488545 (+ 324us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.597742 (+109197us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.597947 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.595148 31165 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:34.624493 31005 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:34.624863 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.620193 30891 rpcz_store.cc:276] 0809 07:33:33.118195 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.477636 (+1359441us) service_pool.cc:227] Handling call
0809 07:33:34.521095 (+ 43459us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.596123 (+ 75028us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.599724 (+ 3601us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.618296 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.627156 30911 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:34.612861 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.628227 31005 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:34.628530 31054 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:34.631894 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.638347 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.650223 30852 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.651364 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:34.651917 30890 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:34.602795 31059 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:34.650187 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.652992 31182 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:34.653303 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:34.654804 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:34.655014 30906 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:34.656363 31005 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:34.656436 30906 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:34.657706 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1338) took 1138 ms. Trace:
I20260809 07:33:34.658084 30906 rpcz_store.cc:276] 0809 07:33:33.518843 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.656040 (+1137197us) service_pool.cc:227] Handling call
0809 07:33:34.656292 (+ 252us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.657502 (+ 1210us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.657658 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.659684 30906 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:34.660243 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1154) took 1114 ms. Trace:
I20260809 07:33:34.661653 30906 rpcz_store.cc:276] 0809 07:33:33.545927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.659109 (+1113182us) service_pool.cc:227] Handling call
0809 07:33:34.659405 (+ 296us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.660031 (+ 626us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.660196 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.663618 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.609390 2446 raft_consensus.cc:2781] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:34.663528 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1274) took 1813 ms. Trace:
I20260809 07:33:34.667794 30853 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.601579 2593 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:34.603864 2635 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:34.664333 31051 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 6.
I20260809 07:33:34.587824 2417 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.630164 2457 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:34.674835 30852 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.588634 2672 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:34.663964 31164 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:34.666692 30906 raft_consensus.cc:1261] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260809 07:33:34.593900 2251 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.661777 31006 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:34.668041 31175 rpcz_store.cc:276] 0809 07:33:32.850034 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:32.850166 (+ 132us) service_pool.cc:227] Handling call
0809 07:33:32.859553 (+ 9387us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:32.871881 (+ 12328us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:32.871913 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:32.874736 (+ 2823us) log.cc:842] Serialized 26 byte log entry
0809 07:33:32.974753 (+100017us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:32.974805 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:32.974880 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:34.459556 (+1484676us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b2400251d38. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091e29cf4 00007f4091e27dfb 00007f4091eb6559 00007f4091eb61e5 00007f4091eb582d 00007f4094b6446e 00007f4094b7bc0c 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551
0809 07:33:34.632050 (+172494us) raft_consensus.cc:1671] finished
0809 07:33:34.632073 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:34.663464 (+ 31391us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5874,"spinlock_wait_cycles":30540032,"thread_start_us":72800,"threads_started":1,"wal-append.queue_time_us":142065}
I20260809 07:33:34.672602 30898 raft_consensus.cc:1261] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:34.657426 2680 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:34.707785 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1148) took 1389 ms. Trace:
I20260809 07:33:34.673005 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:34.629518 2541 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:34.677037 30852 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.665580 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.714373 31055 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:34.571326 2714 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.715544 30896 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260809 07:33:34.715337 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 47642.6us
I20260809 07:33:34.667225 2458 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:34.708166 30898 rpcz_store.cc:276] 0809 07:33:33.318447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.656536 (+1338089us) service_pool.cc:227] Handling call
0809 07:33:34.672325 (+ 15789us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.707499 (+ 35174us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.707712 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:34.598912 2498 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.708184 31164 raft_consensus.cc:595] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.662079 2379 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:34.749125 30891 raft_consensus.cc:1261] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:34.668704 2243 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:34.708025 30906 pending_rounds.cc:76] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Aborting all ops after (but not including) 5
I20260809 07:33:34.720705 30890 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:34.724189 2711 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:34.611059 2659 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.664975 2524 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.670961 2593 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:34.737565 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 60602.7us
I20260809 07:33:34.739061 31164 raft_consensus.cc:603] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:34.630374 2522 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.659108 2669 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:34.667200 2660 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:34.629256 2665 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.754263 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.753162 30906 pending_rounds.cc:97] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260809 07:33:34.675570 2635 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:34.679329 2446 raft_consensus.cc:686] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:34.663805 2548 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260809 07:33:34.773666 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:34.774468 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 101547us
W20260809 07:33:34.775226 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 145703us
I20260809 07:33:34.771054 31058 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.776901 30898 raft_consensus.cc:1261] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:34.607985 2569 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.659196 2683 consensus_queue.cc:260] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.830214 31174 raft_consensus.cc:2379] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28b9965e21374d26b0ac2014c9bafdff in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:34.747998 2541 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:34.650519 2722 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260809 07:33:34.596697 2628 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.630554 2643 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.779860 2417 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:34.777722 2616 log.cc:924] Time spent T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260809 07:33:34.705166 2717 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:34.667219 2527 consensus_queue.cc:1046] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260809 07:33:34.727782 2458 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:34.847119 2458 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:34.849700 2575 consensus_meta.cc:287] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.532s user 0.007s sys 0.003s
I20260809 07:33:34.832427 2658 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:34.847586 2672 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:34.737823 2529 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:34.737588 2457 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:34.789896 30852 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.682157 2326 consensus_queue.cc:1046] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260809 07:33:34.670110 2459 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260809 07:33:34.754866 2675 raft_consensus.cc:2726] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:34.737061 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:34.797426 30890 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:34.797431 30906 raft_consensus.cc:2866] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:34.726606 32139 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:34.803076 31054 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:34.803592 2579 consensus_queue.cc:1046] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:34.804941 2522 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.805330 2251 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.805665 2660 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:34.806960 2725 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260809 07:33:34.808233 2669 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:34.857627 2457 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:34.808969 2665 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:34.814095 30909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1141) took 1497 ms (client timeout 1974 ms). Trace:
W20260809 07:33:34.880409 30909 rpcz_store.cc:269] 0809 07:33:33.316223 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.142095 (+825872us) service_pool.cc:227] Handling call
0809 07:33:34.813930 (+671835us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54001b49d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:34.814037 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":644129536}
I20260809 07:33:34.859616 2595 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.816466 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1322) took 1717 ms. Trace:
I20260809 07:33:34.817322 1856 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:34.819468 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 102596us
I20260809 07:33:34.826663 31005 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:34.827095 30911 raft_consensus.cc:1261] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 07:33:34.777638 2689 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:34.611627 2664 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.831394 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:34.786983 1517 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:34.789995 2593 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.667335 2687 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.841187 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:34.828995 2635 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.844892 2714 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.845099 2527 raft_consensus.cc:3032] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:34.897897 2662 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:34.901312 2527 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:34.901774 2659 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.730048 2626 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.845885 2446 consensus_queue.cc:237] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.902521 2690 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:34.846700 2715 tablet_replica.cc:330] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: stopping tablet replica
I20260809 07:33:34.832615 2541 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:34.903232 2613 log.cc:924] Time spent T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.758s user 0.000s sys 0.003s
I20260809 07:33:34.903522 2541 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:34.904354 2662 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.904937 30890 raft_consensus.cc:1261] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260809 07:33:34.905339 30890 pending_rounds.cc:76] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Aborting all ops after (but not including) 4
I20260809 07:33:34.906728 2498 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.907051 30899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1334) took 1789 ms. Trace:
I20260809 07:33:34.907624 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.908890 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.909960 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.926445 2541 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.928128 2524 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:34.928982 31059 raft_consensus.cc:1261] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:34.839298 2243 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:34.931975 2243 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:34.864413 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 74605.2us
I20260809 07:33:34.864451 30906 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:34.934122 30852 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:34.934916 2251 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:34.928534 2541 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.936450 30906 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:34.936839 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1150) took 1617 ms. Trace:
I20260809 07:33:34.851773 2529 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:34.937773 2529 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:34.939317 31164 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:34.940263 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.853418 2683 raft_consensus.cc:3037] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:34.844005 2561 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:34.869062 2733 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.076s user 0.000s sys 0.001s
I20260809 07:33:34.869415 2660 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.850389 2658 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.866906 32198 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:34.946550 2658 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.870760 2669 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.947024 2651 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:34.874670 2741 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:34.869643 32139 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 165064us
I20260809 07:33:34.858227 2740 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.879289 2628 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:34.938961 2734 log.cc:924] Time spent T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260809 07:33:34.879817 2485 raft_consensus.cc:3032] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:34.813751 2426 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.880959 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.883975 30900 rpcz_store.cc:276] 0809 07:33:33.098687 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.099122 (+ 435us) service_pool.cc:227] Handling call
0809 07:33:33.142207 (+ 43085us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:33.991850 (+849643us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.816263 (+824413us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b54001f1654. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.816420 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1842375552}
I20260809 07:33:34.887478 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:34.887670 30909 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
W20260809 07:33:34.889977 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 63409.8us
I20260809 07:33:34.862843 2722 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.893997 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:34.896271 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:34.880661 2458 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.905539 30890 pending_rounds.cc:97] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260809 07:33:34.979694 30890 raft_consensus.cc:2866] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:34.902288 2664 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.917092 2711 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:34.917266 32199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:34.917542 2689 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.900614 1517 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.964380 2485 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:34.920703 2643 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:34.890548 1856 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.922294 2569 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.926154 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1356) took 1378 ms. Trace:
I20260809 07:33:35.015308 31046 rpcz_store.cc:276] 0809 07:33:33.547465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.547588 (+ 123us) service_pool.cc:227] Handling call
0809 07:33:33.547903 (+ 315us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.326511 (+778608us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.925930 (+599419us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54002404d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.926096 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1708091264}
I20260809 07:33:35.017375 1856 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:34.940363 2672 raft_consensus.cc:3032] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
W20260809 07:33:34.875037 2728 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260809 07:33:34.949723 30906 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:34.950399 31164 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:34.939006 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1147) took 1620 ms. Trace:
I20260809 07:33:34.907428 30899 rpcz_store.cc:276] 0809 07:33:33.117857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.274685 (+156828us) service_pool.cc:227] Handling call
0809 07:33:33.275020 (+ 335us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.201267 (+926247us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.906831 (+705564us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b540045fbd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.906992 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2127905536}
I20260809 07:33:34.952564 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1152) took 1610 ms. Trace:
I20260809 07:33:34.937206 30891 rpcz_store.cc:276] 0809 07:33:33.318952 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.649676 (+1330724us) service_pool.cc:227] Handling call
0809 07:33:34.670490 (+ 20814us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.749512 (+ 79022us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.936604 (+187092us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b5400281254. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.936784 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":180063104}
I20260809 07:33:34.935886 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.021987 30896 rpcz_store.cc:276] 0809 07:33:33.318144 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.654644 (+1336500us) service_pool.cc:227] Handling call
0809 07:33:34.672017 (+ 17373us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.718502 (+ 46485us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.938759 (+220257us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54001fbb54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.938940 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":99299712}
I20260809 07:33:35.022469 30911 rpcz_store.cc:276] 0809 07:33:33.342413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.656739 (+1314326us) service_pool.cc:227] Handling call
0809 07:33:34.656996 (+ 257us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.883527 (+226531us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:34.952306 (+ 68779us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001b53d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:34.952500 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":390459136}
I20260809 07:33:34.878136 2719 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:34.927062 2656 raft_consensus.cc:3032] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:34.960391 30853 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:34.871587 2579 consensus_queue.cc:1225] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer 2c9de56c3fa3487c9dac990675c34be5 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 07:33:34.971124 31048 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:34.972718 31055 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:34.871359 2457 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:34.978675 2638 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.980450 31005 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:34.982643 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:34.985270 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 93262.9us
I20260809 07:33:34.986344 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.033843 31174 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e327743625143d2b7f941cececc7d75"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:34.991401 30900 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:34.964779 2628 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:34.861563 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 67952.1us
I20260809 07:33:35.035128 2628 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.973562 2675 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.036010 2593 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.999943 1517 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:34.985074 2722 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:34.988971 2326 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260809 07:33:34.988790 2658 consensus_queue.cc:1035] T 43645669985b45a0b7e6d18147b28424 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260809 07:33:35.005381 31057 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:35.007695 2458 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.008073 2715 raft_consensus.cc:2229] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260809 07:33:35.008404 2664 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.970440 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
W20260809 07:33:35.023551 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 87740.8us
I20260809 07:33:34.922533 2309 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:35.026249 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 65931.9us
I20260809 07:33:35.018105 2672 raft_consensus.cc:729] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:35.028064 31048 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260809 07:33:35.013432 2683 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:34.963143 2740 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.029479 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.010933 2680 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:35.030539 2426 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.012034 32199 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:35.043879 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 63530.2us
I20260809 07:33:35.087620 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.087466 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:35.087903 31005 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:35.036293 2675 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.088341 32197 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.088963 2569 consensus_queue.cc:1035] T 08675f2b92e541ecba618784caa19bb3 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:35.089551 2722 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.090433 2569 consensus_queue.cc:1035] T 51e40c75108c4a86a26730860edee159 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:35.047360 2643 consensus_queue.cc:1035] T e8bb7f31be3d4e2faf59b0f97f6a803d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:35.077515 32198 raft_consensus.cc:1799] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Handling vote request from an unknown peer 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.092481 32198 raft_consensus.cc:2396] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260809 07:33:35.025671 2659 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.029392 2677 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:35.049510 31051 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [term 6 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:35.025719 2656 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.058574 2719 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:35.058437 2750 consensus_queue.cc:1046] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.011s
I20260809 07:33:35.027832 2711 raft_consensus.cc:686] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:35.069262 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.062011 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:35.048710 2541 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:35.051856 2485 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.047935 2553 consensus_queue.cc:1046] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260809 07:33:35.088209 31164 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.048350 2522 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:34.995652 2689 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.100620 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:35.103308 2715 raft_consensus.cc:2258] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260809 07:33:35.109956 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.107125 2660 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:35.113304 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 51363us
I20260809 07:33:35.112951 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.115078 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.059188 2628 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.116722 31045 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.116662 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.106422 2677 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:35.050199 1517 consensus_queue.cc:1046] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.059s
I20260809 07:33:35.129446 2593 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.118990 2569 consensus_queue.cc:1035] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260809 07:33:35.112676 2605 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:35.131769 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.132956 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.133694 30896 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:35.133571 2527 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.125849 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.126075 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.139477 31170 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:35.126438 30899 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:35.121949 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.060858 2493 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260809 07:33:35.127875 2485 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.068132 2664 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.093847 2683 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.045974 2638 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.124578 2553 consensus_queue.cc:1225] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer 2c9de56c3fa3487c9dac990675c34be5 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260809 07:33:35.159845 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:35.136268 2677 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.160467 31051 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:35.160296 2446 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:35.162194 2446 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.136633 2561 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:35.126377 2522 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.164654 2446 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.138445 32197 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.166112 2522 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.059988 2517 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260809 07:33:35.139712 30911 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:35.166814 2711 consensus_queue.cc:237] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.168006 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:35.168581 30911 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.158078 2664 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.171815 2677 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.174093 2677 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.174470 2656 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.175639 2656 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.084265 2243 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.127887 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:35.135886 2660 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.179476 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.180151 2660 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.117830 2680 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260809 07:33:35.139429 2672 consensus_queue.cc:260] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.183768 2579 consensus_queue.cc:1046] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:35.184765 2579 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:35.185667 2579 consensus_queue.cc:1046] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:35.186476 1517 raft_consensus.cc:3032] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:35.186772 1517 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.187419 1517 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.188156 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.188417 1517 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.189256 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.189821 30896 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:35.190496 2750 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:35.190853 2750 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.191583 2750 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.181079 2527 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:35.192703 2750 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.139127 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.182829 31045 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:35.194628 31164 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.111039 2504 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.142841 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.146080 31054 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:35.148686 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.135648 2575 raft_consensus.cc:3032] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:35.153136 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.118014 2643 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:35.154752 2353 raft_consensus.cc:3032] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:35.120177 2541 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:35.157121 2605 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:35.158041 2326 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.138204 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.208184 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.211513 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.195494 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.197085 30911 raft_consensus.cc:2454] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:35.197299 31170 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:35.193866 30891 raft_consensus.cc:2354] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:35.199285 30853 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:35.074689 2720 consensus_queue.cc:1046] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260809 07:33:35.118820 2658 consensus_queue.cc:1035] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260809 07:33:35.220042 31051 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:35.223188 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.227052 31170 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.229496 30853 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:35.224267 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 71195.5us
I20260809 07:33:35.226264 30900 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:35.229787 31164 raft_consensus.cc:2379] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:35.225775 2326 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.174062 2689 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.231753 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.233515 2683 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.194348 2415 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.234838 2605 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.235253 2638 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.225653 1856 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
W20260809 07:33:35.236183 2758 log.cc:924] Time spent T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.097s user 0.000s sys 0.000s
I20260809 07:33:35.236534 2660 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:35.237174 2651 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:35.206246 2643 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:35.211021 2504 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.278041 31055 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:35.206439 2575 raft_consensus.cc:729] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.238072 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.239065 2529 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:35.278505 2643 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.240482 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.241042 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.240490 2669 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:35.242660 31169 raft_consensus.cc:1261] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260809 07:33:35.240553 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.212327 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.206128 1621 raft_consensus.cc:3032] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:35.155907 2569 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.255177 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.256225 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:35.258404 31170 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:35.165321 2740 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.264474 2541 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.206509 2353 raft_consensus.cc:729] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.266103 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.270246 2326 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.266634 2415 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.273855 30852 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.241288 2662 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:35.262105 2763 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260809 07:33:35.286962 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:35.289094 31169 pending_rounds.cc:76] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:35.303498 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:35.303794 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 47647.6us
I20260809 07:33:35.304292 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.354483 31169 pending_rounds.cc:97] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260809 07:33:35.354673 31054 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.354837 31169 raft_consensus.cc:2866] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:35.279263 2651 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:35.305012 30853 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:35.352186 31006 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.335645 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.304868 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:35.284842 2529 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:35.335268 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.355392 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.355929 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.278709 1856 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:35.356496 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.411298 31174 raft_consensus.cc:595] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
W20260809 07:33:35.411388 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 54964.8us
I20260809 07:33:35.411850 31174 raft_consensus.cc:603] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:35.413152 31005 leader_election.cc:398] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:35.413792 31005 leader_election.cc:401] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:35.414078 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:35.414765 31005 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:35.414501 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 109728us
I20260809 07:33:35.414906 31049 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.393589 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.416092 31051 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.420272 31182 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:35.417213 31048 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:35.421003 31054 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:35.292094 2660 raft_consensus.cc:686] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:35.423684 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.424746 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.292280 1621 raft_consensus.cc:729] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
W20260809 07:33:35.428894 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 76792.4us
I20260809 07:33:35.337644 2527 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:35.430671 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:35.430737 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 156968us
I20260809 07:33:35.431354 31164 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.432034 30852 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.410158 2415 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.425175 31054 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:35.304461 2569 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.362023 2535 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.434890 2637 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:35.306480 2723 consensus_queue.cc:1046] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260809 07:33:35.410614 2575 consensus_queue.cc:260] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.431463 2740 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:35.438398 31049 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.439157 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.440320 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1291) took 1685 ms. Trace:
I20260809 07:33:35.442507 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:35.409085 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:35.466197 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.448151 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:35.427354 2771 log.cc:924] Time spent T c1ccd6862a094ecba07b495d4c772721 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.133s user 0.001s sys 0.000s
I20260809 07:33:35.468374 31169 rpcz_store.cc:276] 0809 07:33:33.754550 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.754710 (+ 160us) service_pool.cc:227] Handling call
0809 07:33:33.755052 (+ 342us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.356432 (+1601380us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.440064 (+ 83632us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54003a02d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:35.440254 (+ 190us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3420479232}
I20260809 07:33:35.450474 31006 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.422286 2326 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:35.491797 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.470996 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.468596 31046 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.441988 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:35.477030 31054 raft_consensus.cc:1799] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Handling vote request from an unknown peer 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.477413 31048 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:35.441360 2529 raft_consensus.cc:2781] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:35.426483 2309 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:35.499737 31054 raft_consensus.cc:2379] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:35.472539 31164 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.477970 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.491125 2575 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.441527 2527 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:35.466027 2723 raft_consensus.cc:3032] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:35.456938 2570 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:35.508895 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.432304 2657 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
W20260809 07:33:35.498543 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 66588.4us
I20260809 07:33:35.522485 30852 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.523676 30852 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.524761 30852 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.522660 1517 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:35.525739 1517 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:35.515261 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.476100 2711 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:35.478889 2659 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:35.517129 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1144) took 2200 ms. Trace:
W20260809 07:33:35.518381 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 213456us
I20260809 07:33:35.540401 30904 rpcz_store.cc:276] 0809 07:33:33.317010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.494721 (+1177711us) service_pool.cc:227] Handling call
0809 07:33:34.541554 (+ 46833us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.597871 (+ 56317us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.516907 (+919036us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b54001a40d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:35.517064 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":106988928}
I20260809 07:33:35.499908 2679 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:35.521063 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.528328 30852 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.468425 2482 consensus_queue.cc:1046] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260809 07:33:35.502844 2309 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:35.420529 2635 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.420317 2457 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.503520 2740 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.477003 2720 consensus_queue.cc:1046] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260809 07:33:35.571137 2746 raft_consensus.cc:3032] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:35.571853 2746 raft_consensus.cc:729] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.464704 1856 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:35.579444 2504 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.498243 2669 consensus_queue.cc:1035] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260809 07:33:35.465891 2660 consensus_queue.cc:237] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.546190 2485 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:35.584930 2457 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.585825 30898 raft_consensus.cc:2379] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 9: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:35.526166 1517 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:35.548697 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.539762 2570 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.590710 2657 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:35.591856 2465 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.552186 30853 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:35.538276 2673 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:35.541656 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.553568 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.516769 2723 raft_consensus.cc:729] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.517406 2527 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.599977 2626 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.599040 2723 consensus_queue.cc:260] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.602923 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1512) took 1017 ms. Trace:
I20260809 07:33:35.517032 2529 raft_consensus.cc:686] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:35.543555 2664 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:35.555975 2309 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:35.548082 2637 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.560525 2687 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.565008 2675 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.465106 2535 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.518680 2569 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.603297 31059 rpcz_store.cc:276] 0809 07:33:34.585633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.652937 (+ 67304us) service_pool.cc:227] Handling call
0809 07:33:34.653181 (+ 244us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.929345 (+276164us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.602659 (+673314us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54000dbbd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:35.602861 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":613384960}
I20260809 07:33:35.608687 1517 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.537118 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.635284 31169 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.611675 31046 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.540194 2561 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:35.623430 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.597518 30904 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:35.464155 2643 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.571930 2720 consensus_queue.cc:1225] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:35.636283 1517 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.592063 2570 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.642607 31059 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:35.645781 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.646533 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.628681 2715 ts_tablet_manager.cc:1905] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 07:33:35.656175 31059 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:35.656962 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.573577 2746 consensus_queue.cc:260] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.658896 30904 raft_consensus.cc:2379] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:35.659703 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.661692 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.662261 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.663022 31164 raft_consensus.cc:2354] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:35.668726 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.669912 2569 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:35.671789 31164 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.653312 2635 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.644726 2561 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.684145 2746 raft_consensus.cc:3037] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.735576 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1160) took 1234 ms. Trace:
I20260809 07:33:35.736104 30890 rpcz_store.cc:276] 0809 07:33:34.500839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.866591 (+365752us) service_pool.cc:227] Handling call
0809 07:33:34.878654 (+ 12063us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.980384 (+101730us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.735284 (+754900us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5400194954. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:35.735501 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":59632896}
I20260809 07:33:35.737308 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.739466 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.740350 30890 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:35.741283 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:35.741461 2775 log.cc:924] Time spent T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.154s user 0.000s sys 0.003s
I20260809 07:33:35.678406 2569 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.746124 2672 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.742013 30890 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:35.759384 31169 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.675980 2529 consensus_queue.cc:237] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.768054 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.766896 30890 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:35.773243 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.774969 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.776733 31169 raft_consensus.cc:2379] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:35.785020 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.776391 31164 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:35.786362 2643 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.786839 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1330) took 1209 ms. Trace:
I20260809 07:33:35.787309 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:43208 (request call id 72) took 1840 ms. Trace:
I20260809 07:33:35.799036 30909 rpcz_store.cc:276] 0809 07:33:33.966383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.973874 (+1007491us) service_pool.cc:227] Handling call
0809 07:33:35.786680 (+812806us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b54000523d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7eaf0 00007f4094f01f2a 00007f4099e3ec6e 00007f4093535807 00007f4093535792 00007f40935356c1 00007f409353563c 00007f40935348f2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:35.787213 (+ 533us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":128636288}
I20260809 07:33:35.796294 31175 rpcz_store.cc:276] 0809 07:33:34.577827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.713561 (+135734us) service_pool.cc:227] Handling call
0809 07:33:34.721492 (+ 7931us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:34.857006 (+135514us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:34.863666 (+ 6660us) log.cc:842] Serialized 14 byte log entry
0809 07:33:34.986969 (+123303us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:34.987027 (+ 58us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:34.987927 (+ 900us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.786640 (+798713us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54003f1454. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:35.786684 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:35.786788 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15954,"prepare.queue_time_us":6969,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":16435,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":10172,"spinlock_wait_cycles":317865600,"thread_start_us":106909,"threads_started":1,"wal-append.queue_time_us":151703}
I20260809 07:33:35.788092 30890 raft_consensus.cc:1226] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:35.790979 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.793684 31057 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:35.839958 31174 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:35.912285 30909 raft_consensus.cc:1226] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260809 07:33:35.792831 2722 raft_consensus.cc:3032] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:35.931190 2722 raft_consensus.cc:729] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:35.910549 2541 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:35.915861 2382 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:35.932935 2541 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.933027 2382 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.924948 2723 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:35.938033 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 385905us
I20260809 07:33:35.938654 2677 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:35.938984 30899 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 07:33:35.939150 2677 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:35.939977 2677 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.942420 2673 raft_consensus.cc:686] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:35.943286 2673 consensus_queue.cc:237] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.945276 31006 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.941937 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.947309 31006 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.927335 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43222 (request call id 1344) took 1183 ms. Trace:
I20260809 07:33:35.948318 30898 rpcz_store.cc:276] 0809 07:33:34.743672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.830188 (+ 86516us) service_pool.cc:227] Handling call
0809 07:33:35.927113 (+1096925us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54003c23d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:35.927249 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1594317696}
I20260809 07:33:35.949847 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.930378 2669 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:35.930698 2773 log.cc:924] Time spent T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.369s user 0.003s sys 0.001s
I20260809 07:33:35.932137 2722 consensus_queue.cc:260] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:35.935104 2382 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.927552 2535 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.951975 31164 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:35.952241 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.962677 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.954079 31175 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:35.963521 30899 raft_consensus.cc:2396] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260809 07:33:35.957008 31005 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.956327 2465 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:35.959021 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 430739us
I20260809 07:33:35.980491 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.959467 31177 raft_consensus.cc:1261] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260809 07:33:35.935108 2541 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:35.964850 31005 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:35.981208 31177 pending_rounds.cc:76] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:35.981503 31177 pending_rounds.cc:97] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260809 07:33:35.981681 31006 leader_election.cc:398] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:35.978519 30853 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:35.977541 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.982576 31005 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:35.961557 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.966476 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.981715 31177 raft_consensus.cc:2866] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:35.983621 31164 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:35.985773 30852 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:35.987133 30852 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:35.987375 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.989650 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:35.991815 2485 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:35.992205 2485 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:35.992517 2485 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:35.994496 2548 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:35.994736 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:35.995344 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:35.995903 31174 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.999207 2561 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:35.999449 2535 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:35.945230 2527 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.000247 2535 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.000492 2382 consensus_queue.cc:1035] T 0cb9e791b482439b936398015bc42aff P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:36.002002 2714 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:36.002453 2714 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:36.002813 2714 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:35.994496 30899 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "12107e38eb6d424abc4f45f16a9a1560"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:36.005162 30899 raft_consensus.cc:595] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Received request to transfer leadership to TS 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:36.005513 30899 raft_consensus.cc:603] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:35.982293 31006 leader_election.cc:401] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:36.020604 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:36.027216 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 48761.8us
I20260809 07:33:36.028494 30853 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:36.030619 30852 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:36.035477 30898 raft_consensus.cc:1261] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:36.036545 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:36.038466 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 78817.7us
I20260809 07:33:36.038128 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:36.039652 31006 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:36.036543 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.041424 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:35.986826 2658 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:35.984773 2722 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:36.046837 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.047667 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:36.059034 30899 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.047801 2679 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:36.049286 2504 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260809 07:33:36.050091 2752 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 1.034s user 0.000s sys 0.014s
I20260809 07:33:36.053025 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260809 07:33:36.065974 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:36.066632 30906 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:35.961041 2738 consensus_queue.cc:1035] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260809 07:33:35.959537 2669 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.023896 2740 raft_consensus.cc:2726] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:36.043928 2427 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:36.010682 2675 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:36.009919 2689 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260809 07:33:36.037110 2773 log.cc:924] Time spent T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260809 07:33:36.069871 30852 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a
I20260809 07:33:36.066900 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:36.043443 2382 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 6. Current term is 7.
I20260809 07:33:36.073453 30853 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:36.061360 2679 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:36.034453 2548 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 07:33:36.075546 31175 pending_rounds.cc:76] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:35.984926 2465 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.072364 2746 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:36.019800 2643 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:36.033397 2569 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.064867 2504 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:36.099913 31006 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:36.063769 30898 raft_consensus.cc:2379] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:36.043373 2687 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260809 07:33:36.102471 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:36.068863 2672 raft_consensus.cc:2781] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:36.081468 2661 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:36.075094 1856 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:36.077194 2675 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 07:33:36.074349 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 53778.1us
I20260809 07:33:36.107323 30899 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:36.151572 30898 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:36.125341 31175 pending_rounds.cc:97] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260809 07:33:36.111835 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.152609 31175 raft_consensus.cc:2866] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:36.128693 2746 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:36.153453 2746 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:36.138875 2535 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:36.111867 30906 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:36.074868 2309 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.154402 2353 consensus_queue.cc:260] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.155438 2353 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.156087 2309 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:36.156772 31169 raft_consensus.cc:2354] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9.
I20260809 07:33:36.158262 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 1314) took 1110 ms. Trace:
I20260809 07:33:36.158586 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1179) took 1005 ms. Trace:
I20260809 07:33:36.158960 30896 rpcz_store.cc:276] 0809 07:33:35.152952 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.293113 (+140161us) service_pool.cc:227] Handling call
0809 07:33:35.352052 (+ 58939us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.430691 (+ 78639us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.430721 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.445466 (+ 14745us) log.cc:842] Serialized 26 byte log entry
0809 07:33:35.777205 (+331739us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:35.777239 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.777310 (+ 71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:36.158382 (+381072us) raft_consensus.cc:1671] finished
0809 07:33:36.158402 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.158531 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":78253,"raft.queue_time_us":82642,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":123,"thread_start_us":269455,"threads_started":1,"wal-append.queue_time_us":563882}
W20260809 07:33:36.160061 2778 log.cc:924] Time spent T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.600s user 0.004s sys 0.012s
W20260809 07:33:36.160806 2790 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260809 07:33:36.158588 31169 rpcz_store.cc:276] 0809 07:33:35.047546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.784811 (+737265us) service_pool.cc:227] Handling call
0809 07:33:36.158088 (+373277us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54003f0f54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:36.158206 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":853267200}
I20260809 07:33:36.162358 2687 consensus_queue.cc:1225] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer a720817fc113446a8ed55e61523b280a log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260809 07:33:36.162884 2687 raft_consensus.cc:3032] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:36.163168 2687 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:36.163571 31054 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:36.161993 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1520) took 1060 ms. Trace:
I20260809 07:33:36.110244 2326 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:36.166070 31055 rpcz_store.cc:276] 0809 07:33:35.101524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.502663 (+401139us) service_pool.cc:227] Handling call
0809 07:33:35.503000 (+ 337us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.503216 (+ 216us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.503241 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.518013 (+ 14772us) log.cc:842] Serialized 26 byte log entry
0809 07:33:35.764130 (+246117us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:35.764163 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.764228 (+ 65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:36.161809 (+397581us) raft_consensus.cc:1671] finished
0809 07:33:36.161829 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.161940 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":199676,"threads_started":1,"wal-append.queue_time_us":517424}
I20260809 07:33:36.175829 30898 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260809 07:33:36.111794 30852 raft_consensus.cc:2670] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260809 07:33:36.168514 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 68689.5us
I20260809 07:33:36.169039 30899 raft_consensus.cc:1226] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:36.161589 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1178) took 1041 ms. Trace:
I20260809 07:33:36.112550 2427 raft_consensus.cc:686] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:36.145675 2776 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.582s user 0.007s sys 0.007s
I20260809 07:33:36.179114 2465 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.149437 2643 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.103714 2762 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:36.180912 2643 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:36.181665 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1364) took 1350 ms. Trace:
I20260809 07:33:36.177178 30891 rpcz_store.cc:276] 0809 07:33:35.120021 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.279690 (+159669us) service_pool.cc:227] Handling call
0809 07:33:35.282924 (+ 3234us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.292382 (+ 9458us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.292413 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.292834 (+ 421us) log.cc:842] Serialized 26 byte log entry
0809 07:33:35.478011 (+185177us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:35.510922 (+ 32911us) log.cc:842] Serialized 14 byte log entry
0809 07:33:35.544172 (+ 33250us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.544242 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:36.159555 (+615313us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:36.161323 (+ 1768us) raft_consensus.cc:1671] finished
0809 07:33:36.161416 (+ 93us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.161536 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":543818,"prepare.queue_time_us":31752,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":57794,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":36850,"spinlock_wait_cycles":85238016,"thread_start_us":184906,"threads_started":1,"wal-append.queue_time_us":266311}
I20260809 07:33:36.183705 31049 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:36.184012 30891 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260809 07:33:36.184859 2651 consensus_meta.cc:287] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff: Time spent flushing consensus metadata: real 0.755s user 0.007s sys 0.002s
I20260809 07:33:36.185513 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1517) took 1084 ms. Trace:
I20260809 07:33:36.185875 31049 rpcz_store.cc:276] 0809 07:33:35.100743 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.467442 (+366699us) service_pool.cc:227] Handling call
0809 07:33:36.183529 (+716087us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.184072 (+ 543us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.185319 (+ 1247us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54000195d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:36.185460 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1643175168}
I20260809 07:33:36.189551 31049 raft_consensus.cc:1226] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:36.185220 2651 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:36.190797 2527 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.190742 31045 rpcz_store.cc:276] 0809 07:33:34.830946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.831103 (+ 157us) service_pool.cc:227] Handling call
0809 07:33:36.181473 (+1350370us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54000195d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:36.181588 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":521670016}
I20260809 07:33:36.192526 31045 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:36.192821 31057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1376) took 1351 ms. Trace:
I20260809 07:33:36.193177 31057 rpcz_store.cc:276] 0809 07:33:34.841365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.059604 (+218239us) service_pool.cc:227] Handling call
0809 07:33:35.086409 (+ 26805us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.840598 (+754189us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.192616 (+352018us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b5400070654. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:36.192765 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1626170240}
I20260809 07:33:36.181583 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1306) took 1605 ms. Trace:
I20260809 07:33:36.195694 31165 rpcz_store.cc:276] 0809 07:33:34.576253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.684037 (+107784us) service_pool.cc:227] Handling call
0809 07:33:34.684457 (+ 420us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:34.713954 (+ 29497us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:34.713982 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:34.721502 (+ 7520us) log.cc:842] Serialized 26 byte log entry
0809 07:33:34.905582 (+184080us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b48000d0758. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4099da07f1 00007f4099d978e7 00007f4099d9777a 00007f4099e8b95f 00007f4099e8b627 00007f4099e9fb2c 00007f4099e9fad8 00007f4099e9fa58 00007f4099e9fa13 00007f4099e9ecbc 00007f4098e20be6
0809 07:33:35.273251 (+367669us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b180025d5c0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411 00007f4094ecd5cf
0809 07:33:35.411124 (+137873us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:35.411158 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.411240 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:36.145102 (+733862us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:36.181382 (+ 36280us) raft_consensus.cc:1671] finished
0809 07:33:36.181403 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.181532 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":75367,"raft.queue_time_us":29335,"raft.run_cpu_time_us":2068,"raft.run_wall_time_us":178561,"spinlock_wait_cycles":147055104,"thread_start_us":137707,"threads_started":1,"wal-append.queue_time_us":828215}
I20260809 07:33:36.196523 2651 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.198884 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1334) took 1304 ms. Trace:
I20260809 07:33:36.199235 31172 rpcz_store.cc:276] 0809 07:33:34.894245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.894405 (+ 160us) service_pool.cc:227] Handling call
0809 07:33:34.894627 (+ 222us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.064134 (+169507us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 07:33:35.079967 (+ 15833us) log.cc:842] Serialized 14 byte log entry
0809 07:33:35.603118 (+523151us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b180037e4c0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b8057b 00007f4094f0388a
0809 07:33:36.102277 (+499159us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:36.102350 (+ 73us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:36.102394 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.198667 (+ 96273us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54002a1cd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:36.198711 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.198832 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":13360,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":210,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7548,"spinlock_wait_cycles":436017920,"thread_start_us":498657,"threads_started":1}
I20260809 07:33:36.201288 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1328) took 1638 ms. Trace:
I20260809 07:33:36.201855 31174 raft_consensus.cc:2454] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
W20260809 07:33:36.138833 2660 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s user 0.006s sys 0.017s
W20260809 07:33:36.206780 2660 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s user 0.006s sys 0.018s
I20260809 07:33:36.207589 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1176) took 1088 ms. Trace:
I20260809 07:33:36.207603 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:36.185793 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 1321) took 1070 ms. Trace:
I20260809 07:33:36.209324 31182 rpcz_store.cc:276] 0809 07:33:35.115807 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.211283 (+ 95476us) service_pool.cc:227] Handling call
0809 07:33:36.185620 (+974337us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54000c3d54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:36.185719 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":478547840}
I20260809 07:33:36.208029 30911 rpcz_store.cc:276] 0809 07:33:35.119527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.223091 (+103564us) service_pool.cc:227] Handling call
0809 07:33:35.223414 (+ 323us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:35.223784 (+ 370us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:35.243332 (+ 19548us) log.cc:842] Serialized 14 byte log entry
0809 07:33:35.409342 (+166010us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:35.409954 (+ 612us) log.cc:842] Serialized 52 byte log entry
0809 07:33:35.410087 (+ 133us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:35.507866 (+ 97779us) log.cc:842] Serialized 14 byte log entry
0809 07:33:35.546237 (+ 38371us) log.cc:842] Serialized 14 byte log entry
0809 07:33:35.925991 (+379754us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b18002178a0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091f484ca 00007f409b629f99 00007f4091e0c990 00007f4091fce864 00007f4091fce155 00007f4094ef13d0 00007f4094f03576 00007f4094f140d7 00007f4094f14058
0809 07:33:35.926184 (+ 193us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.926359 (+ 175us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:35.930091 (+ 3732us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:36.207394 (+277303us) raft_consensus.cc:1671] finished
0809 07:33:36.207416 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.207531 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":528024,"prepare.queue_time_us":30585,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":10207,"raft.queue_time_us":524461,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":15997,"spinlock_wait_cycles":97752704,"thread_start_us":156535,"threads_started":2,"wal-append.queue_time_us":317302}
I20260809 07:33:36.208503 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1323) took 1054 ms. Trace:
I20260809 07:33:36.210572 31164 rpcz_store.cc:276] 0809 07:33:35.154683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.009814 (+855131us) service_pool.cc:227] Handling call
0809 07:33:36.143198 (+133384us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b5800004298. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091f99851 00007f4091f99702 00007f409ae89b09 00007f4099dc5730 00007f4099e8b79b 00007f4099e8b821 00007f4099e4f2b0 00007f4099e3a7f9 00007f4099e3a2a2 00007f4093520684
0809 07:33:36.143413 (+ 215us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.187673 (+ 44260us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.187705 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:36.187743 (+ 38us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:36.187771 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.207185 (+ 19414us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b54000c5b54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:36.207240 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.208446 (+ 1206us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":106050688}
W20260809 07:33:36.214627 2783 log.cc:924] Time spent T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.260s user 0.001s sys 0.000s
W20260809 07:33:36.215580 2787 log.cc:924] Time spent T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.113s user 0.000s sys 0.000s
I20260809 07:33:36.208112 2658 raft_consensus.cc:2726] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:36.217391 2669 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:36.219767 2626 consensus_meta.cc:287] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.619s user 0.007s sys 0.006s
I20260809 07:33:36.220448 31164 raft_consensus.cc:1261] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:36.224483 2745 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.224946 2745 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:36.226135 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 74566.3us
I20260809 07:33:36.226382 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1521) took 1124 ms. Trace:
I20260809 07:33:36.226737 31048 rpcz_store.cc:276] 0809 07:33:35.101761 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.499831 (+398070us) service_pool.cc:227] Handling call
0809 07:33:35.500228 (+ 397us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.500613 (+ 385us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.500643 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.501073 (+ 430us) log.cc:842] Serialized 26 byte log entry
0809 07:33:35.767318 (+266245us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:35.767370 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:35.767438 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:36.226159 (+458721us) raft_consensus.cc:1671] finished
0809 07:33:36.226181 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.226327 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4844,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":6874,"thread_start_us":264640,"threads_started":1,"wal-append.queue_time_us":553827}
I20260809 07:33:36.137959 2504 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.228971 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1524) took 1126 ms. Trace:
I20260809 07:33:36.229321 31051 rpcz_store.cc:276] 0809 07:33:35.108607 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.504476 (+395869us) service_pool.cc:227] Handling call
0809 07:33:35.539020 (+ 34544us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.539727 (+ 707us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.539759 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.552913 (+ 13154us) log.cc:842] Serialized 26 byte log entry
0809 07:33:36.007999 (+455086us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:36.008037 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.008109 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:36.228765 (+220656us) raft_consensus.cc:1671] finished
0809 07:33:36.228798 (+ 33us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.228913 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":358,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":12055,"thread_start_us":374839,"threads_started":1,"wal-append.queue_time_us":585449}
I20260809 07:33:36.125912 2638 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:36.201624 31177 rpcz_store.cc:276] 0809 07:33:34.563077 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.682768 (+119691us) service_pool.cc:227] Handling call
0809 07:33:34.738183 (+ 55415us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.983929 (+1245746us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.201081 (+217152us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003f0a54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:36.201232 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2804869888}
I20260809 07:33:36.113075 2679 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:36.238674 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 61851.6us
I20260809 07:33:36.243714 31006 leader_election.cc:398] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9.
I20260809 07:33:36.244014 31006 leader_election.cc:401] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260809 07:33:36.177325 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 107553us
I20260809 07:33:36.169468 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1518) took 1068 ms. Trace:
I20260809 07:33:36.249320 1856 consensus_queue.cc:1035] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:36.249428 2427 consensus_queue.cc:237] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.250979 1621 consensus_queue.cc:260] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.227289 2791 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:36.252071 1621 raft_consensus.cc:3037] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:36.252169 2791 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.150794 2661 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:36.174767 2675 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.255231 30896 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:36.258991 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:36.239311 2575 consensus_meta.cc:287] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.701s user 0.012s sys 0.000s
I20260809 07:33:36.248440 31058 rpcz_store.cc:276] 0809 07:33:35.100983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.468004 (+367021us) service_pool.cc:227] Handling call
0809 07:33:35.469792 (+ 1788us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.469980 (+ 188us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:35.727279 (+257299us) log.cc:842] Serialized 14 byte log entry
0809 07:33:36.169011 (+441732us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:36.169086 (+ 75us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:36.169155 (+ 69us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.169255 (+ 100us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.169398 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":27891,"prepare.queue_time_us":315218,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":28352,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":64022,"thread_start_us":648497,"threads_started":2}
I20260809 07:33:36.175516 2711 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:36.238814 2638 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:36.261302 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:36.238286 32197 raft_consensus.cc:3037] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.238595 2740 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:36.174749 2672 raft_consensus.cc:686] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:36.284101 31170 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:36.269608 2664 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.270046 2738 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:36.260015 2658 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 9
I20260809 07:33:36.262230 2535 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260809 07:33:36.228263 2657 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260809 07:33:36.272511 2485 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.286681 2791 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:36.272614 2745 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:36.263492 2661 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:36.304010 31120 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:36.338464 31120 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:36.163937 2687 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.279135 2638 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:36.331974 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:36.281863 2515 raft_consensus.cc:3037] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:36.283794 2720 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:36.346963 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.357697 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:36.348996 31120 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:36.347915 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:36.359120 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:36.359930 31172 raft_consensus.cc:2379] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:36.359421 30898 raft_consensus.cc:2379] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 9: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:36.363379 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.408643 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.409490 30911 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:36.410646 30911 raft_consensus.cc:1226] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:36.412266 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
W20260809 07:33:36.412951 2683 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s user 0.015s sys 0.051s
W20260809 07:33:36.413347 2683 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s user 0.015s sys 0.051s
I20260809 07:33:36.413743 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.415041 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:36.299989 2738 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:36.415333 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.416178 31049 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.371248 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:36.441814 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:36.444378 31172 raft_consensus.cc:2354] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28b9965e21374d26b0ac2014c9bafdff for earlier term 6. Current term is 7.
I20260809 07:33:36.306121 2517 consensus_queue.cc:1046] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260809 07:33:36.306430 2575 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260809 07:33:36.332536 2353 consensus_queue.cc:1035] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260809 07:33:36.302729 2664 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.447070 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.333012 2669 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260809 07:33:36.371484 2720 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:36.361032 2723 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:36.349916 2382 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.361770 2745 raft_consensus.cc:2781] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:36.448875 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.452867 2714 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.456298 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:36.545707 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:36.462236 31049 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:36.512465 2465 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:36.411659 2309 consensus_queue.cc:1046] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260809 07:33:36.513324 30898 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 5.
I20260809 07:33:36.360744 2643 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260809 07:33:36.573765 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.575060 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.578703 32197 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:36.579499 2793 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260809 07:33:36.580456 2485 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.447314 2672 consensus_queue.cc:237] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:36.588917 2561 raft_consensus.cc:3032] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:36.589331 2561 raft_consensus.cc:729] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:36.539947 2720 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:36.590210 2561 consensus_queue.cc:260] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.602743 2715 ts_tablet_manager.cc:1918] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.6
W20260809 07:33:36.611819 2722 consensus_meta.cc:287] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.544s user 0.007s sys 0.000s
I20260809 07:33:36.543263 2382 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.617262 2382 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.541495 2723 raft_consensus.cc:686] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:36.426433 2738 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:36.371325 2740 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:36.442534 2691 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:36.411881 2605 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.833s user 0.023s sys 0.031s
W20260809 07:33:36.524461 2800 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.110s user 0.000s sys 0.000s
W20260809 07:33:36.525022 31170 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1349) took 1459 ms (client timeout 1929 ms). Trace:
I20260809 07:33:36.420281 2527 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.361225 2553 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260809 07:33:36.420078 2762 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.411477 2791 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:36.531100 32141 leader_election.cc:398] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28b9965e21374d26b0ac2014c9bafdff for earlier term 6. Current term is 7.
I20260809 07:33:36.677881 2714 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.498986 2627 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.727699 2627 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:36.728765 31005 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.746s user 0.018s sys 0.008s
W20260809 07:33:36.729094 31005 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.746s user 0.018s sys 0.008s
W20260809 07:33:36.729604 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 747043us
I20260809 07:33:36.729806 2627 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.427651 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1379) took 1231 ms. Trace:
I20260809 07:33:36.730409 31046 rpcz_store.cc:276] 0809 07:33:35.196662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.641605 (+444943us) service_pool.cc:227] Handling call
0809 07:33:35.652742 (+ 11137us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.652972 (+ 230us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:36.020276 (+367304us) log.cc:842] Serialized 14 byte log entry
0809 07:33:36.427315 (+407039us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:36.427373 (+ 58us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:36.427420 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.427485 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.427596 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7832,"prepare.queue_time_us":132617,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":75883,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":35755,"thread_start_us":757530,"threads_started":2,"wal-append.queue_time_us":431667}
I20260809 07:33:36.731807 31005 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:36.682580 2605 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.103s user 0.023s sys 0.032s
I20260809 07:33:36.616616 30742 catalog_manager.cc:5495] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 3 to 6, leader changed from a720817fc113446a8ed55e61523b280a (127.28.87.66) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:33:36.463222 2529 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.008s sys 0.050s
I20260809 07:33:36.408325 2635 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:36.599848 2637 consensus_meta.cc:287] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.972s user 0.011s sys 0.000s
W20260809 07:33:36.543455 2673 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s user 0.011s sys 0.047s
W20260809 07:33:36.678980 31170 rpcz_store.cc:269] 0809 07:33:35.065440 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.355142 (+289702us) service_pool.cc:227] Handling call
0809 07:33:36.524801 (+1169659us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54003fded4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:36.524934 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2001197824}
I20260809 07:33:36.698896 31165 raft_consensus.cc:1261] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 07:33:36.542728 2746 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.637840 2715 log.cc:1196] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5: Deleting WAL directory at /tmp/dist-test-taskOwYe5O/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786260685288232-29021-0/minicluster-data/ts-0-root/wals/335fccb907184174b4a6428ce262e7dc
I20260809 07:33:36.679946 2691 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.726825 2791 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:36.715512 32141 leader_election.cc:401] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260809 07:33:36.547142 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.736492 2575 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:36.666476 2379 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:36.735502 30742 catalog_manager.cc:5495] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.810252 30742 catalog_manager.cc:5495] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 3 to 4, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 4 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
W20260809 07:33:36.769400 2673 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s user 0.011s sys 0.048s
I20260809 07:33:36.756361 32197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54908 (request call id 25) took 1442 ms. Trace:
I20260809 07:33:36.739270 2553 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.759060 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:36.715898 2762 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.816411 31165 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 07:33:36.768180 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:36.774979 31059 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s user 0.051s sys 0.055s
W20260809 07:33:36.818903 31059 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s user 0.051s sys 0.055s
I20260809 07:33:36.667131 2740 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.732828 2812 raft_consensus.cc:2741] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260809 07:33:36.628989 2465 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.796612 2746 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:36.798687 31181 raft_consensus.cc:1261] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260809 07:33:36.828049 31181 pending_rounds.cc:76] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 5
I20260809 07:33:36.828444 31181 pending_rounds.cc:97] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260809 07:33:36.828676 31181 raft_consensus.cc:2866] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:36.800217 2638 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.754264 1621 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.790683 2627 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.804313 2791 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.661123 2723 consensus_queue.cc:237] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.811990 30742 catalog_manager.cc:5495] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 4 to 7, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 7 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.753114 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:36.698553 2527 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.757951 2637 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:36.741315 2529 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s user 0.008s sys 0.051s
W20260809 07:33:36.819949 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.679s user 0.000s sys 0.013s
I20260809 07:33:36.822530 31170 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
W20260809 07:33:36.887331 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.746s user 0.000s sys 0.013s
W20260809 07:33:36.825060 31175 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s user 0.007s sys 0.006s
W20260809 07:33:36.889151 31175 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s user 0.007s sys 0.007s
I20260809 07:33:36.829222 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1539) took 1583 ms. Trace:
I20260809 07:33:36.889567 31059 rpcz_store.cc:276] 0809 07:33:35.245382 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.676269 (+430887us) service_pool.cc:227] Handling call
0809 07:33:35.676614 (+ 345us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.676816 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 7
0809 07:33:36.023042 (+346226us) log.cc:842] Serialized 14 byte log entry
0809 07:33:36.828915 (+805873us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:36.828965 (+ 50us) raft_consensus.cc:1631] Marking committed up to 7
0809 07:33:36.829009 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.829065 (+ 56us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:36.829167 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":60574,"prepare.queue_time_us":108975,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":10748,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5014,"thread_start_us":1083240,"threads_started":2}
I20260809 07:33:36.798622 2658 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9.
W20260809 07:33:36.887879 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 746658us
I20260809 07:33:36.892063 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:36.892288 30899 raft_consensus.cc:1226] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:36.842875 30742 catalog_manager.cc:5495] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.893604 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:36.829205 2493 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.898461 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:36.894449 30742 catalog_manager.cc:5495] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 5 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.902228 31170 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:36.828125 2714 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260809 07:33:36.905583 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:36.906198 31177 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:36.907428 31177 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260809 07:33:36.835754 30909 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s user 0.000s sys 0.014s
W20260809 07:33:36.910198 30909 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s user 0.000s sys 0.014s
I20260809 07:33:36.904114 30742 catalog_manager.cc:5495] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 6 to 7, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 7 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.912508 30742 catalog_manager.cc:5495] T 3c486ed5f0d9423992400dfce71ce70f P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.914273 30742 catalog_manager.cc:5495] T f25fc40767244ba0873588714d88e8b6 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 5 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
I20260809 07:33:36.825104 2553 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.814814 2379 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.908532 31051 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:36.841573 2812 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.918181 2548 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.915901 2493 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.915314 31170 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:36.927356 31058 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
W20260809 07:33:36.927872 31169 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.764s user 0.011s sys 0.005s
I20260809 07:33:36.841516 2627 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.941665 2659 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:36.815033 32197 rpcz_store.cc:276] 0809 07:33:35.352408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.401513 (+ 49105us) service_pool.cc:227] Handling call
0809 07:33:36.732793 (+1331280us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400580a54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:36.739149 (+ 6356us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1722947456}
I20260809 07:33:36.944837 30898 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:36.974236 2379 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.985387 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:36.986716 30899 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:36.975471 2715 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43208 (request call id 73) took 3007 ms. Trace:
I20260809 07:33:36.841019 1621 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.909335 2664 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:36.907949 2720 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:36.980520 31169 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.817s user 0.012s sys 0.006s
I20260809 07:33:36.979687 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.017022 31170 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:36.909435 2605 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260809 07:33:36.997929 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:36.991972 2813 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.330s user 0.000s sys 0.001s
W20260809 07:33:36.982039 2809 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.164s user 0.000s sys 0.001s
W20260809 07:33:37.015271 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.744s user 0.011s sys 0.024s
I20260809 07:33:37.021828 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:37.022666 31055 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s user 0.014s sys 0.053s
I20260809 07:33:37.020869 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:37.096100 31055 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s user 0.014s sys 0.053s
W20260809 07:33:37.096582 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.825s user 0.011s sys 0.024s
W20260809 07:33:37.096884 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 825463us
I20260809 07:33:36.990077 30742 catalog_manager.cc:5495] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 5 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:33:37.051679 31045 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s user 0.007s sys 0.015s
W20260809 07:33:36.997205 2427 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s user 0.016s sys 0.054s
W20260809 07:33:37.005118 2569 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s user 0.011s sys 0.023s
W20260809 07:33:37.092965 2483 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.732s user 0.005s sys 0.030s
I20260809 07:33:36.989359 2553 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.111045 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:37.104508 31045 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s user 0.007s sys 0.015s
W20260809 07:33:37.115197 31003 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.688s user 0.024s sys 0.041s
I20260809 07:33:36.979626 2326 consensus_queue.cc:1035] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
W20260809 07:33:37.145882 31003 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.718s user 0.025s sys 0.041s
I20260809 07:33:37.103017 30742 catalog_manager.cc:5495] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 reported cstate change: term changed from 6 to 7, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 7 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: UNKNOWN } } }
W20260809 07:33:37.114111 2427 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s user 0.017s sys 0.055s
I20260809 07:33:36.980095 2812 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.982456 2548 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.991168 2659 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:36.989491 2669 consensus_queue.cc:1046] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260809 07:33:37.117964 2539 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:36.940246 2738 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.015311 2379 consensus_queue.cc:1035] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
W20260809 07:33:37.127254 2569 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s user 0.011s sys 0.023s
I20260809 07:33:37.128247 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.161175 2485 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:37.131145 2483 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.844s user 0.006s sys 0.030s
I20260809 07:33:37.185110 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1357) took 1984 ms. Trace:
W20260809 07:33:37.154513 2807 log.cc:924] Time spent T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.342s user 0.000s sys 0.000s
I20260809 07:33:37.128324 2658 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:37.201511 30900 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.900s user 0.008s sys 0.014s
W20260809 07:33:37.238428 30900 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.937s user 0.008s sys 0.014s
W20260809 07:33:37.127471 31048 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s user 0.005s sys 0.017s
W20260809 07:33:37.239634 31048 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s user 0.005s sys 0.017s
W20260809 07:33:37.172199 30906 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.995s user 0.002s sys 0.023s
W20260809 07:33:37.165258 2461 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.877s user 0.023s sys 0.078s
I20260809 07:33:37.152146 2677 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:37.184068 2745 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.561s user 0.008s sys 0.024s
W20260809 07:33:37.184304 2660 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.011s sys 0.023s
I20260809 07:33:37.220590 30909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1360) took 2019 ms. Trace:
I20260809 07:33:37.224530 30899 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:37.180549 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 62998.2us
I20260809 07:33:37.188073 2427 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.229552 30890 rpcz_store.cc:276] 0809 07:33:35.201038 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.839581 (+638543us) service_pool.cc:227] Handling call
0809 07:33:35.909779 (+ 70198us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.950775 (+ 40996us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.950823 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.951223 (+ 400us) log.cc:842] Serialized 26 byte log entry
0809 07:33:36.372155 (+420932us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:36.372192 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.372260 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:37.184859 (+812599us) raft_consensus.cc:1671] finished
0809 07:33:37.184882 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.185044 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":48011,"raft.queue_time_us":41246,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5858,"thread_start_us":386334,"threads_started":1,"wal-append.queue_time_us":572089}
I20260809 07:33:37.200986 2762 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:37.214855 2817 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.192s user 0.000s sys 0.000s
W20260809 07:33:37.240454 30906 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.063s user 0.003s sys 0.023s
I20260809 07:33:37.238786 2638 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.176329 2691 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.222409 2669 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:37.206346 2605 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.247740 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1355) took 2047 ms. Trace:
I20260809 07:33:37.248523 2457 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:37.250219 31164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1338) took 1306 ms. Trace:
I20260809 07:33:37.236631 2527 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:37.244844 2461 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s user 0.023s sys 0.078s
W20260809 07:33:37.231375 2672 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s user 0.001s sys 0.023s
I20260809 07:33:37.273175 30853 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:37.197898 2465 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:37.250212 2745 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.627s user 0.008s sys 0.024s
W20260809 07:33:37.270661 2660 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s user 0.011s sys 0.023s
I20260809 07:33:37.286228 30890 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 07:33:37.234841 2485 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.248911 2535 raft_consensus.cc:3032] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
W20260809 07:33:37.272537 2675 consensus_meta.cc:287] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.994s user 0.012s sys 0.000s
I20260809 07:33:37.290189 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.309482 30899 raft_consensus.cc:2379] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:37.294976 2527 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:37.275933 2561 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:37.292184 31164 rpcz_store.cc:276] 0809 07:33:35.943746 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.221387 (+277641us) service_pool.cc:227] Handling call
0809 07:33:36.221665 (+ 278us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:36.240931 (+ 19266us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.240963 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:36.241393 (+ 430us) log.cc:842] Serialized 26 byte log entry
0809 07:33:36.733231 (+491838us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:36.733267 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.733381 (+ 114us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:37.250004 (+516623us) raft_consensus.cc:1671] finished
0809 07:33:37.250026 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.250159 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26032,"raft.queue_time_us":24364,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":6183,"thread_start_us":491552,"threads_started":1,"wal-append.queue_time_us":874035}
I20260809 07:33:37.227931 2539 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.288520 30900 rpcz_store.cc:276] 0809 07:33:35.200522 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.784864 (+584342us) service_pool.cc:227] Handling call
0809 07:33:35.785116 (+ 252us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:35.785307 (+ 191us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:35.785629 (+ 322us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.239145 (+1453516us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.247190 (+ 8045us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:37.247310 (+ 120us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:37.247389 (+ 79us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.247514 (+ 125us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.247677 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28961,"prepare.queue_time_us":61863,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":7172,"raft.queue_time_us":181160,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":61305,"thread_start_us":1431100,"threads_started":2,"wal-append.queue_time_us":949525}
I20260809 07:33:37.244984 2658 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.274489 31182 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s user 0.018s sys 0.021s
W20260809 07:33:37.351065 31182 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s user 0.018s sys 0.021s
I20260809 07:33:37.003762 2715 rpcz_store.cc:276] 0809 07:33:33.968097 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:33.970632 (+ 2535us) service_pool.cc:227] Handling call
0809 07:33:34.462341 (+491709us) ts_tablet_manager.cc:1145] Deleting tablet 335fccb907184174b4a6428ce262e7dc
0809 07:33:34.462453 (+ 112us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 07:33:34.846406 (+383953us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001b3ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094f01fe9 00007f4099e89b72 00007f4099eb9e10 00007f4099ea0e0d 00007f4099ea0db9 00007f4099ea0d49 00007f4099ea0d11 00007f4099e9ff3d 00007f409b235204 00007f409b235039
0809 07:33:36.053040 (+1206634us) tablet_metadata.cc:641] Metadata flushed
0809 07:33:36.602334 (+549294us) tablet_metadata.cc:641] Metadata flushed
0809 07:33:36.975343 (+373009us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":79853,"spinlock_wait_cycles":293234560,"tablet-delete.queue_time_us":374849,"thread_start_us":199987,"threads_started":1}
I20260809 07:33:37.270726 30909 rpcz_store.cc:276] 0809 07:33:35.201502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.917061 (+715559us) service_pool.cc:227] Handling call
0809 07:33:35.917426 (+ 365us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:35.960858 (+ 43432us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:35.960890 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:35.979297 (+ 18407us) log.cc:842] Serialized 26 byte log entry
0809 07:33:36.146834 (+167537us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b180037a140. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fb3345 00007f409adeaa39 00007f4091fd08bb 00007f4091fcecbf 00007f4091fd0e22 00007f4094b7a5bb 00007f4094b80063 00007f4094b80411 00007f4094ecd5cf
0809 07:33:36.910796 (+763962us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:36.910836 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.910904 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:37.220223 (+309319us) raft_consensus.cc:1671] finished
0809 07:33:37.220255 (+ 32us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.220522 (+ 267us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":43005,"raft.queue_time_us":59852,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4290,"spinlock_wait_cycles":9286400,"thread_start_us":763780,"threads_started":1,"wal-append.queue_time_us":1215917}
W20260809 07:33:37.257969 31120 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.713s user 0.000s sys 0.025s
W20260809 07:33:37.305003 2672 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s user 0.002s sys 0.023s
I20260809 07:33:37.270843 2677 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.332864 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.279928 2427 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.309011 2664 raft_consensus.cc:2781] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
W20260809 07:33:37.340981 31121 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.007s sys 0.015s
I20260809 07:33:37.312573 2627 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:37.356462 31120 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.811s user 0.001s sys 0.026s
I20260809 07:33:37.311431 2535 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:37.308988 2745 raft_consensus.cc:686] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:37.313179 1621 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:37.358646 31121 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.837s user 0.007s sys 0.015s
I20260809 07:33:37.315804 2638 consensus_queue.cc:1035] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260809 07:33:37.293354 2669 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.289829 2605 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.317090 2720 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.358616 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1324) took 2153 ms. Trace:
I20260809 07:33:37.358705 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:37.427771 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 69147.3us
W20260809 07:33:37.329363 32141 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.526s user 0.000s sys 0.024s
I20260809 07:33:37.320220 2465 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:37.416682 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 960398us
I20260809 07:33:37.417675 2852 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:37.358146 30849 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.796s user 0.024s sys 0.024s
I20260809 07:33:37.418099 30724 catalog_manager.cc:4841] TS 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): tablet 335fccb907184174b4a6428ce262e7dc (table test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]) successfully deleted
I20260809 07:33:37.423341 2485 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.322497 2675 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.449551 2828 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.091s user 0.000s sys 0.000s
I20260809 07:33:37.450105 2675 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.433158 30853 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:37.409006 2664 raft_consensus.cc:686] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:37.439286 30849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.031s user 0.024s sys 0.024s
I20260809 07:33:37.435200 2672 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:37.443154 2553 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.443898 2539 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.438355 2745 consensus_queue.cc:237] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.444679 2677 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.445415 2461 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260809 07:33:37.446125 2858 raft_consensus.cc:482] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:37.447011 2825 log.cc:924] Time spent T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.124s user 0.000s sys 0.000s
I20260809 07:33:37.425477 2659 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.447321 2669 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.435500 1621 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.450603 30898 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:37.427110 2661 consensus_meta.cc:287] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.063s user 0.012s sys 0.000s
I20260809 07:33:37.515431 2661 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.516413 2840 log.cc:924] Time spent T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
W20260809 07:33:37.517256 2811 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.714s user 0.002s sys 0.000s
I20260809 07:33:37.517711 2661 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.455561 2465 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.518455 30891 raft_consensus.cc:2354] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9.
I20260809 07:33:37.519690 2465 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.338822 2527 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.415432 2831 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:37.454923 31164 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:37.452472 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.535454 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.535748 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.536532 31182 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:37.537433 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:37.435587 32141 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.631s user 0.001s sys 0.024s
I20260809 07:33:37.429741 2836 raft_consensus.cc:3032] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:37.451152 2820 raft_consensus.cc:2781] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
W20260809 07:33:37.451869 2504 consensus_meta.cc:287] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.205s user 0.009s sys 0.003s
W20260809 07:33:37.451131 2822 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.210s user 0.000s sys 0.000s
I20260809 07:33:37.437348 2483 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:37.452386 2738 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.436064 2812 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.514138 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.426621 31169 rpcz_store.cc:276] 0809 07:33:35.205708 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.162676 (+956968us) service_pool.cc:227] Handling call
0809 07:33:36.162961 (+ 285us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.163160 (+ 199us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 07:33:37.015855 (+852695us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.357030 (+341175us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:37.357080 (+ 50us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:37.357128 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.358443 (+ 1315us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.358561 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1113419,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":465,"raft.run_wall_time_us":68480,"thread_start_us":1192379,"threads_started":2}
I20260809 07:33:37.442243 31120 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:37.555727 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:37.556319 31169 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:37.521983 30904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1183) took 1976 ms (client timeout 2109 ms). Trace:
I20260809 07:33:37.523398 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:37.459453 2852 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.557158 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 115208us
I20260809 07:33:37.528019 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1307) took 3204 ms. Trace:
I20260809 07:33:37.557672 31165 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 10: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:37.528674 31051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1399) took 1112 ms. Trace:
W20260809 07:33:37.556731 30904 rpcz_store.cc:269] 0809 07:33:35.545274 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:35.661458 (+116184us) service_pool.cc:227] Handling call
0809 07:33:37.521770 (+1860312us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400209254. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:37.521919 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2663138560}
I20260809 07:33:37.539806 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.465700 2672 raft_consensus.cc:504] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.549890 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.569310 30899 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:37.553417 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1395) took 1143 ms. Trace:
I20260809 07:33:37.465842 2553 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.527179 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1401) took 1082 ms. Trace:
W20260809 07:33:37.578610 31177 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s user 0.004s sys 0.059s
I20260809 07:33:37.557706 31181 rpcz_store.cc:276] 0809 07:33:34.323544 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:34.324316 (+ 772us) service_pool.cc:227] Handling call
0809 07:33:34.324627 (+ 311us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.829050 (+2504423us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.527758 (+698708us) spinlock_profiling.cc:243] Waited 2.47 s on lock 0x7b54003f1e54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:37.527930 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5689082112}
I20260809 07:33:37.580080 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:37.538597 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.540328 2831 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.562453 2517 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P 12107e38eb6d424abc4f45f16a9a1560 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:37.562754 32197 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:37.477895 2858 raft_consensus.cc:504] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.549235 2483 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:37.563855 2852 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.566968 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1361) took 2364 ms. Trace:
I20260809 07:33:37.543695 2836 raft_consensus.cc:729] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:37.575079 31048 rpcz_store.cc:276] 0809 07:33:36.410331 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.411934 (+ 1603us) service_pool.cc:227] Handling call
0809 07:33:36.449416 (+ 37482us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.449619 (+ 203us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 07:33:36.497649 (+ 48030us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.546221 (+1048572us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:37.546276 (+ 55us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:37.546344 (+ 68us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.546407 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.553358 (+ 6951us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":48336,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2121,"raft.queue_time_us":69351,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":2918,"thread_start_us":1048183,"threads_started":2,"wal-append.queue_time_us":913402}
I20260809 07:33:37.579140 31058 rpcz_store.cc:276] 0809 07:33:36.444430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.533681 (+ 89251us) service_pool.cc:227] Handling call
0809 07:33:37.526999 (+993318us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b5400339ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:37.527114 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":823559936}
I20260809 07:33:37.609212 31175 rpcz_store.cc:276] 0809 07:33:35.202150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.054264 (+852114us) service_pool.cc:227] Handling call
0809 07:33:36.075365 (+ 21101us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:36.153195 (+ 77830us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.153225 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:36.153638 (+ 413us) log.cc:842] Serialized 26 byte log entry
0809 07:33:36.896008 (+742370us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:36.896048 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:36.896199 (+ 151us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:37.566732 (+670533us) raft_consensus.cc:1671] finished
0809 07:33:37.566769 (+ 37us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.566906 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":742093,"threads_started":1,"wal-append.queue_time_us":1042756}
I20260809 07:33:37.610642 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.611624 31045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1391) took 1669 ms. Trace:
I20260809 07:33:37.581100 31182 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:37.558456 31051 rpcz_store.cc:276] 0809 07:33:36.416707 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.574787 (+158080us) service_pool.cc:227] Handling call
0809 07:33:37.528517 (+953730us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54000db1d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:37.528618 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":765505792}
I20260809 07:33:37.547019 2665 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:37.615679 30909 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:37.581760 31177 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s user 0.004s sys 0.059s
I20260809 07:33:37.582044 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.616654 2831 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.588402 31169 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:37.591811 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:37.617352 31049 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:37.593400 2846 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:37.618738 2846 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.620842 2846 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.612056 2483 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:37.624629 31049 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:37.626598 2427 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.612368 31045 rpcz_store.cc:276] 0809 07:33:35.941887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.193157 (+251270us) service_pool.cc:227] Handling call
0809 07:33:36.193455 (+ 298us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:36.248801 (+ 55346us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.248837 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:36.249440 (+ 603us) log.cc:842] Serialized 52 byte log entry
0809 07:33:37.145693 (+896253us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:37.541105 (+395412us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.546582 (+ 5477us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.546654 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:37.611419 (+ 64765us) raft_consensus.cc:1671] finished
0809 07:33:37.611440 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.611566 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":62048,"prepare.queue_time_us":417104,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":57768,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":2575,"thread_start_us":1247973,"threads_started":2,"wal-append.queue_time_us":1032310}
I20260809 07:33:37.578161 31120 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:37.544467 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 1.05049e+06us
I20260809 07:33:37.540932 1621 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.575958 2679 consensus_meta.cc:287] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.309s user 0.012s sys 0.000s
I20260809 07:33:37.477097 2535 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.644737 2679 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.645751 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.616534 2553 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.646847 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:37.647730 2672 leader_election.cc:288] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:37.649417 2819 log.cc:924] Time spent T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.423s user 0.000s sys 0.001s
I20260809 07:33:37.562516 2855 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28b9965e21374d26b0ac2014c9bafdff for earlier term 6. Current term is 7.
I20260809 07:33:37.650300 2866 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:37.516127 2664 consensus_queue.cc:237] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.635082 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 56992.8us
I20260809 07:33:37.578646 2691 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.651548 2691 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:37.652864 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:37.581316 31059 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:37.653605 30891 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 10: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:37.549192 2504 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.655525 1621 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.655938 31175 raft_consensus.cc:1261] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 07:33:37.654047 2691 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:37.641033 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:37.642511 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.644126 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.657303 31051 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:37.656908 31045 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:37.657792 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.646891 2679 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.617550 2836 consensus_queue.cc:260] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.658229 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.657482 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:37.661468 31005 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:37.640235 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:37.655807 2829 log.cc:924] Time spent T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.211s user 0.000s sys 0.001s
I20260809 07:33:37.662953 31182 raft_consensus.cc:1261] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260809 07:33:37.676569 31182 pending_rounds.cc:76] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 3
I20260809 07:33:37.676826 31182 pending_rounds.cc:97] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260809 07:33:37.677085 31182 raft_consensus.cc:2866] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:37.661846 31164 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:37.638454 2841 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 07:33:37.604384 2740 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.603087 2868 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:37.610033 2658 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.674855 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.688418 31049 raft_consensus.cc:2396] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 07:33:37.603223 2835 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260809 07:33:37.675652 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:37.691262 31169 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:37.691690 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:37.680989 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.655627 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:37.668668 31045 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 10.
I20260809 07:33:37.684597 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.691852 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.694693 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.672674 2669 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.678736 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:37.694778 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.700278 31046 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 10: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:37.696017 31059 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:37.692060 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:37.674140 2687 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:37.697221 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.697894 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:37.702219 31048 raft_consensus.cc:2379] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 9: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:37.679147 1621 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:37.695533 30891 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:37.706163 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.706920 31165 raft_consensus.cc:2379] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 9: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:37.685669 2836 raft_consensus.cc:3037] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:37.679591 2714 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:37.682780 2465 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:37.708750 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:37.683261 2605 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.680235 2812 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:37.735127 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:37.720242 31005 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:37.721654 31120 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:37.724239 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.736492 31049 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:37.698729 2841 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:37.737404 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.737623 31005 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:37.738111 31049 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:37.738977 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.740128 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.739563 31049 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:37.741547 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:37.740206 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 192036us
I20260809 07:33:37.742897 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:37.676910 2872 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260809 07:33:37.707719 2575 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 07:33:37.710937 1621 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:37.829294 2828 log.cc:924] Time spent T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260809 07:33:37.879848 1621 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:37.780047 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:37.780073 2527 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.879825 2677 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:37.778556 2842 log.cc:924] Time spent T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.214s user 0.000s sys 0.001s
I20260809 07:33:37.778488 2637 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:37.880064 2575 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:37.881806 2575 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:37.881706 2637 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.881018 2677 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.751753 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:37.730788 2465 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.885166 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.885792 2677 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.884728 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.784861 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.880739 31051 raft_consensus.cc:3037] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:37.782130 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.888482 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.889057 30891 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:37.890584 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.891130 30891 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:37.891351 1621 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:37.870816 2863 log.cc:924] Time spent T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.197s user 0.000s sys 0.000s
I20260809 07:33:37.747397 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:37.894285 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.894613 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.895001 31177 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 11.
W20260809 07:33:37.898411 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 152661us
I20260809 07:33:37.728528 2517 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:37.839469 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1186) took 2293 ms. Trace:
I20260809 07:33:37.901096 30906 rpcz_store.cc:276] 0809 07:33:35.546494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.164341 (+617847us) service_pool.cc:227] Handling call
0809 07:33:36.164644 (+ 303us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:36.165192 (+ 548us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.165219 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:36.176675 (+ 11456us) log.cc:842] Serialized 26 byte log entry
0809 07:33:37.278150 (+1101475us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:37.278428 (+ 278us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.295045 (+ 16617us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.295120 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:37.655250 (+360130us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:37.815771 (+160521us) raft_consensus.cc:1671] finished
0809 07:33:37.839164 (+ 23393us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.839385 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44303,"prepare.queue_time_us":146927,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4700,"raft.queue_time_us":16872,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":11004,"spinlock_wait_cycles":5406592,"thread_start_us":1101103,"threads_started":1,"wal-append.queue_time_us":1268200}
I20260809 07:33:37.903321 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.709226 2669 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.906699 30891 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:37.900535 31165 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:37.691241 2868 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.782846 2504 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:37.858162 2561 consensus_meta.cc:287] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.515s user 0.008s sys 0.002s
I20260809 07:33:37.746045 2820 raft_consensus.cc:686] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:37.725697 2714 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.764688 2858 leader_election.cc:288] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:37.710764 32197 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:37.746557 31172 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:37.749065 2841 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:37.908116 31051 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:37.864425 2483 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:37.923794 31055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1545) took 2652 ms. Trace:
I20260809 07:33:37.840184 2720 raft_consensus.cc:482] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.944418 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.894934 2677 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:37.729040 2873 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 07:33:37.849644 2871 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.104s user 0.000s sys 0.000s
W20260809 07:33:37.851032 32139 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.66:36075 timed out after 1.640s (SENT) [suppressed 56 similar messages]
I20260809 07:33:37.931456 30853 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:37.933579 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 190749us
I20260809 07:33:37.744484 2658 raft_consensus.cc:2726] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:37.894250 1621 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:37.940038 31164 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:37.747012 2791 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
W20260809 07:33:37.941015 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 203479us
I20260809 07:33:37.941848 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:37.840525 2846 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260809 07:33:37.925069 2569 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.004s sys 0.013s
I20260809 07:33:37.931777 2675 consensus_queue.cc:1035] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 07:33:37.943831 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:37.949780 2895 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:37.927459 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:37.950678 31049 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:37.927151 2382 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:37.928056 2517 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:37.958664 31120 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:37.944818 31055 rpcz_store.cc:276] 0809 07:33:35.271502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.232348 (+960846us) service_pool.cc:227] Handling call
0809 07:33:36.232647 (+ 299us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.232849 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 07:33:36.264214 (+ 31365us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.738680 (+1474466us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.923445 (+184765us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:37.923501 (+ 56us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:37.923552 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.923621 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:37.923734 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":217627,"prepare.queue_time_us":160386,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":73,"raft.queue_time_us":243890,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":12318,"thread_start_us":1450728,"threads_started":3,"wal-append.queue_time_us":1044168}
I20260809 07:33:37.942780 2745 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:38.016782 2569 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.732s user 0.005s sys 0.013s
I20260809 07:33:38.015475 31051 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 10: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:38.008939 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.021745 30853 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:37.886166 2465 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.024533 31054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36496 (request call id 1393) took 1614 ms. Trace:
I20260809 07:33:38.032287 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.040606 31055 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 6.
I20260809 07:33:38.041344 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.041790 31055 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:38.042727 31055 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:37.986267 31059 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:37.946532 2879 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:38.044653 31054 rpcz_store.cc:276] 0809 07:33:36.409727 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.411401 (+ 1674us) service_pool.cc:227] Handling call
0809 07:33:36.411677 (+ 276us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:36.843031 (+431354us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.843064 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:36.843105 (+ 41us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:36.843134 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.024299 (+1181165us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400030cd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:38.024349 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:38.024478 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":991559424}
I20260809 07:33:38.054646 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08675f2b92e541ecba618784caa19bb3" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:38.049268 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 90658.5us
I20260809 07:33:38.014963 2846 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:37.949775 2677 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:37.981886 2673 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:38.072974 31169 raft_consensus.cc:1261] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:38.079530 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:38.074292 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.033165 2791 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:38.080705 31051 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:38.036340 2637 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.034014 2517 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:37.946602 2635 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.124s
I20260809 07:33:38.066026 2846 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:37.947865 2841 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.069250 2679 raft_consensus.cc:2781] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:38.072084 2677 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:38.109730 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:37.960100 2561 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:38.053661 2879 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:37.955790 2720 raft_consensus.cc:504] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:38.088809 2504 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:38.038848 2745 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.066088 2527 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260809 07:33:38.038097 2895 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.062603 2820 consensus_queue.cc:237] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.268177 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.084105 2673 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:38.408530 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:38.410698 31006 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:38.184507 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 120495us
I20260809 07:33:38.185617 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:38.283120 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:38.144702 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.128504 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.271652 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:38.416989 31058 raft_consensus.cc:3037] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 9
W20260809 07:33:38.451808 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 168307us
I20260809 07:33:38.425633 31058 raft_consensus.cc:2454] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:38.448428 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.454867 31055 raft_consensus.cc:2379] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:38.454914 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:38.261009 2687 consensus_meta.cc:287] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.553s user 0.006s sys 0.001s
I20260809 07:33:38.456588 31058 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
W20260809 07:33:38.252592 2888 log.cc:924] Time spent T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
W20260809 07:33:38.251221 2907 log.cc:924] Time spent T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260809 07:33:38.458880 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.463922 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:38.403299 2879 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:38.465567 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:38.465956 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.144958 2912 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:38.480280 31055 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:38.245812 2493 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:38.417004 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.468314 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:38.468601 31174 raft_consensus.cc:1261] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260809 07:33:38.255246 2483 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.470643 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:38.508037 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 46261.2us
I20260809 07:33:38.243995 2669 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.248790 2841 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:38.195899 2868 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.418399 2673 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:38.516583 2723 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:38.517143 2723 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.518842 2569 consensus_queue.cc:1035] T 335fccb907184174b4a6428ce262e7dc P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:33:38.519181 2723 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:38.519738 2669 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:38.519959 2723 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:38.520840 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:38.475507 2689 consensus_queue.cc:1035] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260809 07:33:38.465056 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.522360 2895 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.522634 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:38.481055 31045 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:38.524391 2720 leader_election.cc:288] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:38.525517 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:38.519057 2529 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:38.526633 2529 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.525669 31045 raft_consensus.cc:1261] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:38.458009 2880 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260809 07:33:38.531502 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 426199us
I20260809 07:33:38.256265 2575 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.514364 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:38.509389 31174 pending_rounds.cc:76] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Aborting all ops after (but not including) 4
I20260809 07:33:38.529981 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:38.535413 31169 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 5 FOLLOWER]: Advancing to term 7
I20260809 07:33:38.535682 31174 pending_rounds.cc:97] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260809 07:33:38.536504 31174 raft_consensus.cc:2866] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:38.464977 2679 raft_consensus.cc:686] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:38.410041 2561 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.541560 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:38.550199 31051 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
W20260809 07:33:38.467492 32139 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 616470us
I20260809 07:33:38.535252 31177 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:38.407683 2504 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.556629 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:38.214071 2791 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.521194 2687 raft_consensus.cc:3032] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:38.578099 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:38.578819 31177 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:38.531232 2880 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:38.507649 2812 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:38.525861 2745 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:38.563511 31169 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:38.521138 2569 consensus_queue.cc:1035] T 5ed1309ae2b34b6085b236210555d243 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260809 07:33:38.539270 2691 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.559163 31006 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:38.649367 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 90261.7us
I20260809 07:33:38.605971 31051 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:38.546492 2689 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:38.546648 2868 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.549463 2539 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:38.580579 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1210) took 1042 ms. Trace:
I20260809 07:33:38.670990 30853 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:38.686599 30890 rpcz_store.cc:276] 0809 07:33:37.537757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.540322 (+ 2565us) service_pool.cc:227] Handling call
0809 07:33:37.542707 (+ 2385us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:37.748485 (+205778us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:37.748515 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:37.780790 (+ 32275us) log.cc:842] Serialized 26 byte log entry
0809 07:33:37.950274 (+169484us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:37.950308 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.950421 (+ 113us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:38.580356 (+629935us) raft_consensus.cc:1671] finished
0809 07:33:38.580377 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:38.580516 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5127,"raft.queue_time_us":9875,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":14946,"thread_start_us":373829,"threads_started":2,"wal-append.queue_time_us":327743}
I20260809 07:33:38.573426 2834 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.580055 2493 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.695940 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.701097 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.701947 30890 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:38.573040 2912 raft_consensus.cc:686] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:38.577474 2687 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:38.616225 2880 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:38.575852 2658 consensus_queue.cc:1046] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.135s
I20260809 07:33:38.704892 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.708204 31006 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:38.671016 2891 raft_consensus.cc:482] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:38.562520 2675 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.134s
I20260809 07:33:38.691123 2689 raft_consensus.cc:686] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:38.697412 2669 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260809 07:33:38.720114 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 251878us
I20260809 07:33:38.677024 2812 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.732292 31005 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:38.682226 2691 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.739396 31005 leader_election.cc:398] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9.
I20260809 07:33:38.694283 2868 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.758939 2868 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.759569 2868 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:38.759884 2868 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:38.760197 2868 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:38.761776 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1416) took 1037 ms. Trace:
I20260809 07:33:38.762141 31059 rpcz_store.cc:276] 0809 07:33:37.724394 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.739080 (+ 14686us) service_pool.cc:227] Handling call
0809 07:33:38.761576 (+1022496us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400018bd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:38.761702 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":234289024}
I20260809 07:33:38.721719 2895 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.703114 2834 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.743471 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:38.720379 2891 raft_consensus.cc:504] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.744009 2912 consensus_queue.cc:237] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.779824 2575 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:38.744683 2504 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:38.735873 2720 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.738397 2896 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:38.747295 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:38.743903 2561 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:38.750622 2909 log.cc:924] Time spent T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
I20260809 07:33:38.751085 31049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36496 (request call id 1415) took 1027 ms. Trace:
I20260809 07:33:38.796627 31049 rpcz_store.cc:276] 0809 07:33:37.724117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.741305 (+ 17188us) service_pool.cc:227] Handling call
0809 07:33:38.750894 (+1009589us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54000302d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:38.751024 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":479255936}
I20260809 07:33:38.751833 2762 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:38.769238 31181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1382) took 1314 ms. Trace:
I20260809 07:33:38.797840 31181 rpcz_store.cc:276] 0809 07:33:37.454853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.580550 (+125697us) service_pool.cc:227] Handling call
0809 07:33:37.609187 (+ 28637us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:37.630439 (+ 21252us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:37.630472 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:37.630514 (+ 42us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:37.630542 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.769013 (+1138471us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b54003d0754. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:38.769059 (+ 46us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:38.769181 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":48321792}
I20260809 07:33:38.723223 2669 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:38.696722 2539 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:38.800712 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 346332us
I20260809 07:33:38.801333 30890 raft_consensus.cc:2379] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 7: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:38.802241 31120 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:38.742765 31005 leader_election.cc:401] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:38.773942 31051 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:38.774663 2541 raft_consensus.cc:3032] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:38.774860 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:38.829252 31059 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:38.712397 2493 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.777700 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:38.739326 2711 raft_consensus.cc:2781] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:38.696949 2679 consensus_queue.cc:237] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.735530 2529 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:38.843123 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 103889us
I20260809 07:33:38.847558 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:38.848834 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 46677.9us
I20260809 07:33:38.849546 31120 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:38.802603 2669 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Advancing to term 12
I20260809 07:33:38.777577 2895 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.800421 2896 raft_consensus.cc:686] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:38.863620 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:38.762900 2687 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.876153 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:38.873334 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:38.796741 2720 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.840510 2541 raft_consensus.cc:729] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
W20260809 07:33:38.872902 30898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1203) took 1820 ms (client timeout 1599 ms). Trace:
I20260809 07:33:38.875074 31051 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:38.840103 2711 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:38.800321 2762 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.866977 2675 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:38.873078 30900 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260809 07:33:38.873754 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1340) took 2929 ms. Trace:
I20260809 07:33:38.843988 2504 raft_consensus.cc:482] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:38.856273 2836 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9.
I20260809 07:33:38.882809 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:38.911791 30898 rpcz_store.cc:269] 0809 07:33:37.052893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.110718 (+ 57825us) service_pool.cc:227] Handling call
0809 07:33:38.872674 (+1761956us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54001f2554. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:38.872831 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":735670016}
I20260809 07:33:38.867183 2660 raft_consensus.cc:482] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:38.921461 31174 rpcz_store.cc:276] 0809 07:33:35.944247 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.237415 (+293168us) service_pool.cc:227] Handling call
0809 07:33:36.245233 (+ 7818us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:38.537415 (+2292182us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.873515 (+336100us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54003890d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:38.873681 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5109659904}
I20260809 07:33:38.876330 2575 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:38.930470 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c486ed5f0d9423992400dfce71ce70f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.937974 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.938657 30898 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:38.939459 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:38.813838 2689 consensus_queue.cc:237] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.935009 31048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1555) took 1211 ms. Trace:
W20260809 07:33:38.933087 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 50375.4us
I20260809 07:33:38.946791 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:38.927791 31177 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:38.915756 2687 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:38.913707 2852 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:38.951587 30898 raft_consensus.cc:1226] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260809 07:33:38.973928 30898 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:38.952471 31177 raft_consensus.cc:1261] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:38.919094 2711 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:38.944932 31048 rpcz_store.cc:276] 0809 07:33:37.723300 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.725068 (+ 1768us) service_pool.cc:227] Handling call
0809 07:33:37.741218 (+ 16150us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:37.929287 (+188069us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:37.929323 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:37.929810 (+ 487us) log.cc:842] Serialized 26 byte log entry
0809 07:33:38.095450 (+165640us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:38.112909 (+ 17459us) log.cc:842] Serialized 14 byte log entry
0809 07:33:38.113317 (+ 408us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.113430 (+ 113us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:38.934805 (+821375us) raft_consensus.cc:1671] finished
0809 07:33:38.934829 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:38.934951 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":187692,"prepare.queue_time_us":19138,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":190953,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"spinlock_wait_cycles":72249728,"thread_start_us":165260,"threads_started":1,"wal-append.queue_time_us":598695}
I20260809 07:33:38.884831 2926 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:38.992100 2635 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 07:33:38.972757 2852 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:38.932775 2675 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.933192 2504 raft_consensus.cc:504] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:39.001751 2664 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.345s user 0.026s sys 0.021s
I20260809 07:33:38.944940 2660 raft_consensus.cc:504] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:38.949028 2575 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.002357 2812 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.013511 2852 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.031885 2539 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:39.037709 2846 consensus_meta.cc:287] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.907s user 0.010s sys 0.001s
W20260809 07:33:39.027856 2664 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.375s user 0.026s sys 0.021s
I20260809 07:33:38.987819 2541 consensus_queue.cc:260] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:39.040736 2921 log.cc:924] Time spent T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.260s user 0.001s sys 0.000s
I20260809 07:33:38.987352 2896 consensus_queue.cc:237] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.040112 2920 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260809 07:33:39.045549 2918 log.cc:924] Time spent T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.195s user 0.001s sys 0.001s
I20260809 07:33:39.040273 2924 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:39.058410 2920 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:39.045595 2928 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:39.131259 2762 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.039911 2930 consensus_queue.cc:1035] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.165s
I20260809 07:33:39.139837 2880 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.059890 2895 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.146787 2928 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.045339 2846 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.160281 2691 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.155740 2720 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.060954 2920 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:39.163355 2714 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.174292 2846 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
W20260809 07:33:39.154091 2936 log.cc:924] Time spent T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.112s user 0.000s sys 0.000s
W20260809 07:33:39.179256 2941 log.cc:924] Time spent T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.129s user 0.000s sys 0.000s
I20260809 07:33:39.060212 2924 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:39.182678 2924 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:39.162789 2956 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.094s user 0.000s sys 0.001s
I20260809 07:33:39.196970 2928 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.160540 2868 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.201989 2868 leader_election.cc:288] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:39.211848 2879 consensus_meta.cc:287] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.683s user 0.007s sys 0.004s
I20260809 07:33:39.212601 2879 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.142145 2880 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.243371 2920 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:39.216184 2879 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.240602 2924 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.248405 2924 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.254767 2920 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
W20260809 07:33:39.287159 30853 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.601s user 0.006s sys 0.014s
W20260809 07:33:39.288120 30853 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.602s user 0.006s sys 0.014s
I20260809 07:33:39.301069 31169 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
W20260809 07:33:39.303076 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 356212us
I20260809 07:33:39.304641 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1222) took 1220 ms. Trace:
I20260809 07:33:39.305138 30898 rpcz_store.cc:276] 0809 07:33:38.084213 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.974612 (+890399us) service_pool.cc:227] Handling call
0809 07:33:39.303479 (+328867us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.303957 (+ 478us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.304003 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.304062 (+ 59us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:39.304089 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.304185 (+ 96us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.304400 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 07:33:39.306975 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 461530us
I20260809 07:33:39.323243 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1392) took 1267 ms. Trace:
I20260809 07:33:39.323828 31182 rpcz_store.cc:276] 0809 07:33:38.055089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.057936 (+ 2847us) service_pool.cc:227] Handling call
0809 07:33:38.058298 (+ 362us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:38.058674 (+ 376us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:38.058702 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:38.066523 (+ 7821us) log.cc:842] Serialized 26 byte log entry
0809 07:33:38.459554 (+393031us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:38.459593 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.459660 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.322558 (+862898us) raft_consensus.cc:1671] finished
0809 07:33:39.322599 (+ 41us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.322920 (+ 321us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":345134,"spinlock_wait_cycles":79350144,"thread_start_us":354520,"threads_started":1,"wal-append.queue_time_us":622081}
I20260809 07:33:39.337798 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:39.338951 32198 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:39.305892 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:39.347234 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.350796 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1393) took 1295 ms. Trace:
I20260809 07:33:39.351259 31175 rpcz_store.cc:276] 0809 07:33:38.055389 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.058518 (+ 3129us) service_pool.cc:227] Handling call
0809 07:33:38.086139 (+ 27621us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:38.598650 (+512511us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:38.598689 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:38.599228 (+ 539us) log.cc:842] Serialized 26 byte log entry
0809 07:33:38.868373 (+269145us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.322162 (+453789us) log.cc:842] Serialized 14 byte log entry
0809 07:33:39.350188 (+ 28026us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.350332 (+ 144us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.350420 (+ 88us) raft_consensus.cc:1671] finished
0809 07:33:39.350430 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.350624 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49430,"prepare.queue_time_us":169499,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":319812,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":39834,"thread_start_us":1202462,"threads_started":3,"wal-append.queue_time_us":423000}
I20260809 07:33:39.353611 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:39.368515 31005 leader_election.cc:302] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.372097 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1226) took 1271 ms. Trace:
I20260809 07:33:39.372450 30898 rpcz_store.cc:276] 0809 07:33:38.100660 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.370833 (+1270173us) service_pool.cc:227] Handling call
0809 07:33:39.371382 (+ 549us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.371752 (+ 370us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.371776 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.371831 (+ 55us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.371868 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.371920 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.372027 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:39.383215 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.385358 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:39.385921 31175 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:39.385414 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.426488 2504 leader_election.cc:288] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:39.422148 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 116541us
W20260809 07:33:39.434374 30849 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.534s user 0.008s sys 0.009s
W20260809 07:33:39.442864 30849 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.544s user 0.009s sys 0.010s
I20260809 07:33:39.446712 2762 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:39.447685 2762 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.449028 2493 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.449236 2866 raft_consensus.cc:482] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:39.449994 2493 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:39.450968 2836 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:39.451939 2738 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:39.452440 2873 raft_consensus.cc:2726] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:39.453193 2575 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.450117 2866 raft_consensus.cc:504] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.454183 2910 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:39.453348 2535 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.454882 2910 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:39.455576 2910 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:39.420013 2561 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:39.458802 2714 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:39.459650 2675 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.461333 2541 raft_consensus.cc:3037] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.445930 31005 leader_election.cc:302] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.249575 2852 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:39.463496 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 92320.9us
I20260809 07:33:39.464597 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:39.465684 2852 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.466725 2879 raft_consensus.cc:2726] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:39.467692 2866 leader_election.cc:288] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:39.464852 31121 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.507s user 0.003s sys 0.016s
I20260809 07:33:39.469725 30898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:43208 (request call id 77) took 1355 ms. Trace:
W20260809 07:33:39.471777 30904 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s user 0.009s sys 0.013s
W20260809 07:33:39.473680 30904 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s user 0.010s sys 0.014s
I20260809 07:33:39.477187 2457 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:39.467657 2879 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:39.478943 2879 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:39.451864 2836 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.481858 2660 leader_election.cc:288] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.482659 31170 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
W20260809 07:33:39.478600 30852 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.512s user 0.001s sys 0.009s
W20260809 07:33:39.483559 30852 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.572s user 0.001s sys 0.009s
W20260809 07:33:39.485179 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 609040us
W20260809 07:33:39.487128 2673 consensus_meta.cc:287] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.971s user 0.011s sys 0.000s
I20260809 07:33:39.487630 2673 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.488981 30899 raft_consensus.cc:2354] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:39.489769 2673 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.479348 2879 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.481191 2836 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.470153 30898 rpcz_store.cc:276] 0809 07:33:38.114200 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.373348 (+1259148us) service_pool.cc:227] Handling call
0809 07:33:39.469644 (+ 96296us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:39.491479 31005 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.499890 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:39.459709 2714 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.510619 2846 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:39.511200 2635 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:39.506479 32198 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
W20260809 07:33:39.513266 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 72956.4us
I20260809 07:33:39.503291 2762 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.451123 2493 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.520183 2836 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:39.511436 2714 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.522888 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:39.515636 2673 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:39.525990 2673 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.528148 32199 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260809 07:33:39.528810 2923 log.cc:924] Time spent T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.593s user 0.006s sys 0.021s
I20260809 07:33:39.525556 2691 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260809 07:33:39.533797 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 67665.9us
I20260809 07:33:39.511327 2846 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:39.535715 2677 consensus_meta.cc:287] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 1.074s user 0.009s sys 0.006s
I20260809 07:33:39.512869 2635 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.540699 2635 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.540942 30890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1216) took 1460 ms. Trace:
I20260809 07:33:39.541433 30890 rpcz_store.cc:276] 0809 07:33:38.080500 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.702595 (+622095us) service_pool.cc:227] Handling call
0809 07:33:39.540448 (+837853us) spinlock_profiling.cc:243] Waited 94.3 ms on lock 0x7b5400052154. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.540673 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":216914688}
I20260809 07:33:39.542213 2575 raft_consensus.cc:482] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:39.533986 2691 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:39.522598 2836 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.524996 2866 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:39.546622 31165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 1374) took 1767 ms. Trace:
I20260809 07:33:39.549297 31165 rpcz_store.cc:276] 0809 07:33:37.779565 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.784587 (+ 5022us) service_pool.cc:227] Handling call
0809 07:33:39.546226 (+1761639us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b54000c4754. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.546405 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":30841088}
I20260809 07:33:39.545482 31046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36492 (request call id 1566) took 1639 ms. Trace:
I20260809 07:33:39.543491 30904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1378) took 1983 ms. Trace:
W20260809 07:33:39.548523 31177 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.505s user 0.011s sys 0.002s
I20260809 07:33:39.537129 2714 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:39.544988 2669 consensus_meta.cc:287] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.668s user 0.005s sys 0.005s
I20260809 07:33:39.554484 31172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1347) took 3047 ms. Trace:
I20260809 07:33:39.550251 31046 rpcz_store.cc:276] 0809 07:33:37.905537 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.050618 (+145081us) service_pool.cc:227] Handling call
0809 07:33:38.763108 (+712490us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:38.763298 (+ 190us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:38.763322 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:38.763360 (+ 38us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:38.763388 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.763443 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.544992 (+781549us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400018bd0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.545178 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1561558784}
I20260809 07:33:39.557587 30853 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:39.558907 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.545841 2846 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.548153 2691 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:39.555754 31172 rpcz_store.cc:276] 0809 07:33:36.454679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.456816 (+ 2137us) service_pool.cc:227] Handling call
0809 07:33:36.500550 (+ 43734us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:37.940940 (+1440390us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.446278 (+1505338us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400185454. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:39.502457 (+ 56179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2865197824}
I20260809 07:33:39.556016 31005 leader_election.cc:302] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.563325 2493 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.568142 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.568575 30906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1211) took 1489 ms. Trace:
I20260809 07:33:39.543049 2575 raft_consensus.cc:504] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.571246 31182 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:39.575623 31175 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:39.576206 2948 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:39.576647 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:39.577507 30909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1208) took 2040 ms (client timeout 1864 ms). Trace:
W20260809 07:33:39.577960 30909 rpcz_store.cc:269] 0809 07:33:37.537091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.537214 (+ 123us) service_pool.cc:227] Handling call
0809 07:33:39.577081 (+2039867us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b54001937d4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.577252 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":178498432}
I20260809 07:33:39.570537 30906 rpcz_store.cc:276] 0809 07:33:38.079171 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.081823 (+ 2652us) service_pool.cc:227] Handling call
0809 07:33:38.082185 (+ 362us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:38.415887 (+333702us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:38.415917 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:38.451887 (+ 35970us) log.cc:842] Serialized 26 byte log entry
0809 07:33:38.735947 (+284060us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:38.736091 (+ 144us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:38.736161 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.524953 (+788792us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b44000232cc. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4091e15f11 00007f4091ee3a95 00007f4091ee44b2 00007f4094b8c511 00007f4094b87e1a 00007f4094b7bbee 00007f4094b7af36 00007f4094b7ab58 00007f4094b84551 00007f4094b84509 00007f4094b84499
0809 07:33:39.559883 (+ 34930us) raft_consensus.cc:1671] finished
0809 07:33:39.559904 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.568379 (+ 8475us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":77655,"raft.queue_time_us":333309,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":21600,"spinlock_wait_cycles":5665664,"thread_start_us":283758,"threads_started":1,"wal-append.queue_time_us":459686}
I20260809 07:33:39.577064 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:39.578540 31170 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45794 (request call id 1409) took 1473 ms (client timeout 1721 ms). Trace:
W20260809 07:33:39.579710 31170 rpcz_store.cc:269] 0809 07:33:38.104663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.408269 (+303606us) service_pool.cc:227] Handling call
0809 07:33:39.573491 (+1165222us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400185e54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.578474 (+ 4983us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2467132160}
I20260809 07:33:39.580159 30896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1382) took 2006 ms. Trace:
I20260809 07:33:39.580389 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.580991 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.581322 31174 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:39.581769 31170 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:39.584358 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1414) took 1048 ms. Trace:
I20260809 07:33:39.587323 31182 rpcz_store.cc:276] 0809 07:33:38.536155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.582307 (+1046152us) service_pool.cc:227] Handling call
0809 07:33:39.582758 (+ 451us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.584058 (+ 1300us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.584081 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.584138 (+ 57us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.584167 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.584213 (+ 46us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.584306 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:39.577543 2866 raft_consensus.cc:2726] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:39.588587 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1412) took 1054 ms. Trace:
I20260809 07:33:39.589891 30900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1379) took 2028 ms. Trace:
I20260809 07:33:39.588876 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1396) took 1532 ms. Trace:
I20260809 07:33:39.590927 2866 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:39.589916 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 68216.3us
I20260809 07:33:39.591890 2866 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.569729 2669 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.588490 30896 rpcz_store.cc:276] 0809 07:33:37.573956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.940591 (+366635us) service_pool.cc:227] Handling call
0809 07:33:39.578757 (+1638166us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.579122 (+ 365us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.579145 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.579213 (+ 68us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.579244 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.579319 (+ 75us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.579914 (+ 595us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54001937d0. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.580002 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3727911808}
W20260809 07:33:39.593196 2517 consensus_meta.cc:287] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 1.443s user 0.002s sys 0.015s
I20260809 07:33:39.593669 2517 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.590756 31169 rpcz_store.cc:276] 0809 07:33:38.056202 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.646167 (+589965us) service_pool.cc:227] Handling call
0809 07:33:38.646530 (+ 363us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.301934 (+655404us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.588560 (+286626us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b54003f1454. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:39.588786 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1503794560}
I20260809 07:33:39.579993 31175 raft_consensus.cc:2379] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:39.578531 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:39.589660 2711 consensus_meta.cc:287] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.596s user 0.001s sys 0.013s
I20260809 07:33:39.552161 30904 rpcz_store.cc:276] 0809 07:33:37.560125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.560295 (+ 170us) service_pool.cc:227] Handling call
0809 07:33:37.560630 (+ 335us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:37.560941 (+ 311us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:37.560970 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:38.838430 (+1277460us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b2c00012038. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f40938dce11 00007f40938ec442 00007f4094f34c11 00007f4094ef958a 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772
0809 07:33:38.863285 (+ 24855us) log.cc:842] Serialized 26 byte log entry
0809 07:33:39.476178 (+612893us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.476559 (+ 381us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.478019 (+ 1460us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.543051 (+ 65032us) raft_consensus.cc:1671] finished
0809 07:33:39.543070 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.543400 (+ 330us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5369,"raft.queue_time_us":45942,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":27005,"spinlock_wait_cycles":27234048,"thread_start_us":559754,"threads_started":1,"wal-append.queue_time_us":641585}
I20260809 07:33:39.600252 30911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43218 (request call id 1198) took 2654 ms. Trace:
I20260809 07:33:39.591598 30900 rpcz_store.cc:276] 0809 07:33:37.561173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.561288 (+ 115us) service_pool.cc:227] Handling call
0809 07:33:38.872851 (+1311563us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:38.921099 (+ 48248us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.589566 (+668467us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001f2550. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef8a09 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.589732 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3008852352}
I20260809 07:33:39.590196 31174 rpcz_store.cc:276] 0809 07:33:38.534521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.584501 (+1049980us) service_pool.cc:227] Handling call
0809 07:33:39.584927 (+ 426us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.585261 (+ 334us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.585290 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.585358 (+ 68us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.585407 (+ 49us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.585460 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.588505 (+ 3045us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:39.605423 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.607606 2665 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:39.553668 2677 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.610356 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.607553 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.611799 2836 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.607514 31005 leader_election.cc:302] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.614064 2834 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.611060 31175 raft_consensus.cc:1226] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:39.595654 2575 leader_election.cc:288] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.618664 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:39.614406 2948 raft_consensus.cc:686] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:39.615357 2955 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:39.614382 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.601152 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:39.611129 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.620584 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.595913 2866 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:39.615720 2908 log.cc:924] Time spent T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.924s user 0.031s sys 0.018s
I20260809 07:33:39.613965 2669 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.615670 2673 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.638015 2665 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:39.661666 2575 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:39.653014 30898 raft_consensus.cc:1789] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: voting while tombstoned based on last-logged opid 5.6
I20260809 07:33:39.683404 2575 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.617472 2835 consensus_queue.cc:1035] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260809 07:33:39.683996 2879 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.688544 2691 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.637521 30911 rpcz_store.cc:276] 0809 07:33:36.946150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.946259 (+ 109us) service_pool.cc:227] Handling call
0809 07:33:36.946598 (+ 339us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:36.946918 (+ 320us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:36.946943 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:36.947410 (+ 467us) log.cc:842] Serialized 26 byte log entry
0809 07:33:37.315721 (+368311us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:37.907500 (+591779us) log.cc:842] Serialized 14 byte log entry
0809 07:33:37.949628 (+ 42128us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:37.949707 (+ 79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.593065 (+1643358us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b18002ed320. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f409ae708b1 00007f409ae7d661 00007f4091fd1638 00007f4091fd4b11 00007f4091fd4ac9 00007f4091fd4a59 00007f4091fd4a21 00007f4091fd3d1d 00007f409b235204 00007f409b235039 00007f4091fb3dc6
0809 07:33:39.595848 (+ 2783us) raft_consensus.cc:1671] finished
0809 07:33:39.599827 (+ 3979us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.600021 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":36846,"prepare.queue_time_us":148742,"prepare.run_cpu_time_us":6899,"prepare.run_wall_time_us":46741,"raft.queue_time_us":78428,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":24032,"spinlock_wait_cycles":9201408,"thread_start_us":810136,"threads_started":2,"wal-append.queue_time_us":563109}
W20260809 07:33:39.690832 2933 log.cc:924] Time spent T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.651s user 0.003s sys 0.009s
W20260809 07:33:39.692799 31057 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43890 (request call id 7) took 3493 ms (client timeout 1640 ms). Trace:
W20260809 07:33:39.693851 30899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43218 (request call id 1214) took 1613 ms (client timeout 1774 ms). Trace:
I20260809 07:33:39.695222 30891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43222 (request call id 1388) took 2116 ms. Trace:
I20260809 07:33:39.697634 2548 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.704016 30745 catalog_manager.cc:5495] T 335fccb907184174b4a6428ce262e7dc P 28b9965e21374d26b0ac2014c9bafdff reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
W20260809 07:33:39.708801 31164 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45802 (request call id 1368) took 2028 ms (client timeout 2216 ms). Trace:
W20260809 07:33:39.711113 2906 log.cc:924] Time spent T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.997s user 0.001s sys 0.012s
W20260809 07:33:39.735572 30899 rpcz_store.cc:269] 0809 07:33:38.080019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.522380 (+442361us) service_pool.cc:227] Handling call
0809 07:33:39.693593 (+1171213us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b540045fbd4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.693786 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2220369280}
I20260809 07:33:39.614684 2659 raft_consensus.cc:482] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:39.668860 2665 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:39.712850 2962 log.cc:924] Time spent T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.654s user 0.000s sys 0.012s
I20260809 07:33:39.697537 2677 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.712951 2948 consensus_queue.cc:237] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.639400 2714 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.732482 30898 raft_consensus.cc:2454] T 335fccb907184174b4a6428ce262e7dc P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
W20260809 07:33:39.737598 31164 rpcz_store.cc:269] 0809 07:33:37.680582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.680703 (+ 121us) service_pool.cc:227] Handling call
0809 07:33:39.708436 (+2027733us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400389ad4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.708618 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":567052032}
W20260809 07:33:39.735219 31057 rpcz_store.cc:269] 0809 07:33:36.198735 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:36.207362 (+ 8627us) service_pool.cc:227] Handling call
0809 07:33:39.446409 (+3239047us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5400311b20. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094e89730 00007f4094efceff 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86
0809 07:33:39.692342 (+245933us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540034add4. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094efd2ae 00007f4099e3ac27 00007f4093522c04 00007f4093522b92 00007f4093522ac1 00007f4093522a3c 00007f4093521cf2 00007f409332394c 00007f4093322d86 00007f409332272f
0809 07:33:39.692533 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2641417344}
W20260809 07:33:39.744620 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 89104.1us
I20260809 07:33:39.752199 31169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1384) took 1037 ms. Trace:
I20260809 07:33:39.754225 31174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1386) took 1012 ms. Trace:
I20260809 07:33:39.755260 31174 rpcz_store.cc:276] 0809 07:33:38.737316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.649105 (+911789us) service_pool.cc:227] Handling call
0809 07:33:39.715443 (+ 66338us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.746159 (+ 30716us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.746191 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.749945 (+ 3754us) raft_consensus.cc:1631] Marking committed up to 6
0809 07:33:39.749988 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.750048 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.750154 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 07:33:39.753108 31169 rpcz_store.cc:276] 0809 07:33:38.736649 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.595102 (+858453us) service_pool.cc:227] Handling call
0809 07:33:39.595489 (+ 387us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:39.595849 (+ 360us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:39.605156 (+ 9307us) log.cc:842] Serialized 14 byte log entry
0809 07:33:39.745172 (+140016us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.746596 (+ 1424us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.746640 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.746702 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.746840 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":190,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":34,"raft.queue_time_us":2912,"thread_start_us":135042,"threads_started":1}
I20260809 07:33:39.735685 30891 rpcz_store.cc:276] 0809 07:33:37.578185 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:37.950247 (+372062us) service_pool.cc:227] Handling call
0809 07:33:38.025295 (+ 75048us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:38.845510 (+820215us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:38.845542 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:38.845684 (+ 142us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:38.845719 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.694794 (+849075us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54003b9f54. stack: 00007f4091f9f033 00007f4091f9ef30 00007f40918fd534 00007f409b6098b4 00007f409b609849 00007f4094e7e471 00007f4094ef9f51 00007f4094ef899e 00007f4099e3a2e9 00007f4093520684 00007f4093520612 00007f4093520541 00007f40935204bc 00007f409351f772 00007f409332394c 00007f4093322d86
0809 07:33:39.694857 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.695063 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":75184,"raft.queue_time_us":122211,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":40228,"spinlock_wait_cycles":993770880,"thread_start_us":312433,"threads_started":1}
I20260809 07:33:39.656141 2711 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.714366 2955 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.754525 2517 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.738909 31170 raft_consensus.cc:2932] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:39.770884 2483 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260809 07:33:39.753880 2866 raft_consensus.cc:2726] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:39.774775 31177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45794 (request call id 1411) took 1241 ms. Trace:
W20260809 07:33:39.783545 2926 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.719s user 0.003s sys 0.016s
W20260809 07:33:39.783957 2926 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.765s user 0.003s sys 0.017s
I20260809 07:33:39.784231 2926 raft_consensus.cc:686] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:39.784549 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:39.787340 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:39.788542 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 170082us
I20260809 07:33:39.790894 2834 consensus_queue.cc:1046] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:39.791159 30853 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:39.793107 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.791963 1856 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:39.794097 1856 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
W20260809 07:33:39.786443 2687 consensus_meta.cc:287] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.753s user 0.007s sys 0.006s
I20260809 07:33:39.785504 2926 consensus_queue.cc:237] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.795503 2891 leader_election.cc:288] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.795732 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.794509 1856 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.797556 2711 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.799389 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.800215 2714 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.795094 30899 raft_consensus.cc:1261] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260809 07:33:39.805644 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 198325us
W20260809 07:33:39.812570 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 148733us
I20260809 07:33:39.815869 2910 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.792191 2834 raft_consensus.cc:3032] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:39.778612 2659 raft_consensus.cc:504] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.793131 2483 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 07:33:39.776780 31177 rpcz_store.cc:276] 0809 07:33:38.533830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.981293 (+447463us) service_pool.cc:227] Handling call
0809 07:33:38.981553 (+ 260us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 07:33:38.981737 (+ 184us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 07:33:39.560839 (+579102us) log.cc:842] Serialized 14 byte log entry
0809 07:33:39.744115 (+183276us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 07:33:39.752344 (+ 8229us) raft_consensus.cc:1631] Marking committed up to 4
0809 07:33:39.763165 (+ 10821us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.768764 (+ 5599us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.774717 (+ 5953us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":63460,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":6506,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3766,"thread_start_us":669680,"threads_started":2}
I20260809 07:33:39.831485 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.831223 30906 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260809 07:33:39.831840 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.821797 1856 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.834918 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.832675 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.825142 32199 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260809 07:33:39.843117 30898 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:39.829216 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:39.843416 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.850865 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:39.854036 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:39.848984 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.854861 30899 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:39.820420 2834 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:39.857152 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:39.833076 2483 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:39.818282 2910 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.859764 31006 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:39.824150 2659 leader_election.cc:288] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.863579 30909 raft_consensus.cc:2379] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:39.836836 2575 leader_election.cc:288] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.835023 2691 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.835500 2879 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.828467 2665 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.862712 1856 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.862947 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.862164 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.865666 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.863857 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.861750 2483 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:39.871493 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:39.814568 2892 consensus_queue.cc:1046] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:39.860487 2517 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:39.824194 2711 consensus_queue.cc:1046] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260809 07:33:39.871187 30904 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.890545 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.895629 30906 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:39.905294 30909 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
I20260809 07:33:39.910879 30890 raft_consensus.cc:2379] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate 2c9de56c3fa3487c9dac990675c34be5 in this term.
W20260809 07:33:39.910748 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 81637.9us
I20260809 07:33:39.911885 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.914721 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:39.915366 31172 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.916630 31164 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:39.917068 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.917657 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:39.917773 2665 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.918228 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:39.919561 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:39.922868 31174 raft_consensus.cc:2379] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:39.923601 2517 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.925688 2517 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.925807 31177 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
I20260809 07:33:39.927371 31177 raft_consensus.cc:1261] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:39.929301 31005 leader_election.cc:398] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:39.930415 31006 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:39.931772 31174 raft_consensus.cc:1226] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:39.932446 31177 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260809 07:33:39.933122 31174 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:39.935405 31164 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:39.937311 31182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1383) took 1228 ms. Trace:
I20260809 07:33:39.937736 31182 rpcz_store.cc:276] 0809 07:33:38.708957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.635881 (+926924us) service_pool.cc:227] Handling call
0809 07:33:39.735909 (+100028us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:39.779928 (+ 44019us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.779957 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:39.810047 (+ 30090us) log.cc:842] Serialized 26 byte log entry
0809 07:33:39.857309 (+ 47262us) raft_consensus.cc:1631] Marking committed up to 5
0809 07:33:39.857348 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.857419 (+ 71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.937111 (+ 79692us) raft_consensus.cc:1671] finished
0809 07:33:39.937133 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.937258 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5182,"raft.queue_time_us":6737,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5312,"thread_start_us":19949,"threads_started":1,"wal-append.queue_time_us":98281}
I20260809 07:33:39.860787 2891 consensus_queue.cc:1046] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260809 07:33:39.940629 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 79399.9us
I20260809 07:33:39.943084 1856 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:39.943243 30852 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:39.946379 30904 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
W20260809 07:33:39.939293 2988 log.cc:924] Time spent T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260809 07:33:39.955613 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:39.929868 2483 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:39.961812 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 88089.6us
I20260809 07:33:39.962479 1856 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:39.916020 2866 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:39.970533 2955 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:39.970950 2866 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:39.963907 31006 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:39.965204 31177 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:39.956177 2879 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:39.967842 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.970350 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.974433 2866 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.974805 2382 raft_consensus.cc:2781] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:39.976392 2382 raft_consensus.cc:686] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [term 9 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:39.977435 2382 consensus_queue.cc:237] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.979858 2872 raft_consensus.cc:2781] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:39.980083 31006 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:39.987443 2866 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:39.989136 2866 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:39.989773 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:39.980921 2929 consensus_queue.cc:1046] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260809 07:33:39.960126 2892 consensus_queue.cc:1225] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer 12107e38eb6d424abc4f45f16a9a1560 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 07:33:39.993640 31170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1382) took 1285 ms. Trace:
I20260809 07:33:39.994243 30906 raft_consensus.cc:3032] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:39.994992 30906 raft_consensus.cc:729] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:39.996833 31172 raft_consensus.cc:2454] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:39.994048 31170 rpcz_store.cc:276] 0809 07:33:38.708641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.584276 (+875635us) service_pool.cc:227] Handling call
0809 07:33:39.584964 (+ 688us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 07:33:39.585697 (+ 733us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 07:33:39.585718 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 07:33:39.606954 (+ 21236us) log.cc:842] Serialized 520 byte log entry
0809 07:33:39.732964 (+126010us) raft_consensus.cc:1631] Marking committed up to 7
0809 07:33:39.951287 (+218323us) log.cc:842] Serialized 14 byte log entry
0809 07:33:39.951457 (+ 170us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.951506 (+ 49us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:39.993423 (+ 41917us) raft_consensus.cc:1671] finished
0809 07:33:39.993441 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:39.993581 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":417,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":2942,"spinlock_wait_cycles":4992,"thread_start_us":125169,"threads_started":1,"wal-append.queue_time_us":175191}
I20260809 07:33:40.000558 2929 raft_consensus.cc:3032] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:40.000952 2929 raft_consensus.cc:729] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:40.001808 2929 consensus_queue.cc:260] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.003090 2929 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:39.997340 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:39.982138 2872 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:40.009898 2955 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.010449 2955 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:40.010914 2955 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.014886 1856 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.015292 1856 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:40.015617 1856 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.019742 31170 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:39.994532 2879 raft_consensus.cc:686] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:40.021230 31175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45802 (request call id 1387) took 1250 ms. Trace:
I20260809 07:33:40.021992 2879 consensus_queue.cc:237] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.021629 31175 rpcz_store.cc:276] 0809 07:33:38.770553 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:39.651794 (+881241us) service_pool.cc:227] Handling call
0809 07:33:39.746249 (+ 94455us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 07:33:39.805412 (+ 59163us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 07:33:39.843613 (+ 38201us) log.cc:842] Serialized 14 byte log entry
0809 07:33:39.951751 (+108138us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 07:33:39.963863 (+ 12112us) log.cc:842] Serialized 26 byte log entry
0809 07:33:39.963989 (+ 126us) raft_consensus.cc:1631] Marking committed up to 3
0809 07:33:39.964018 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 07:33:39.964079 (+ 61us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 07:33:40.021006 (+ 56927us) raft_consensus.cc:1671] finished
0809 07:33:40.021023 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 07:33:40.021168 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6949,"prepare.queue_time_us":38155,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":31679,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":2439,"thread_start_us":57245,"threads_started":1,"wal-append.queue_time_us":151988}
I20260809 07:33:40.020677 31170 raft_consensus.cc:595] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.024195 31170 raft_consensus.cc:603] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:40.024899 2834 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
W20260809 07:33:40.027045 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 47053.6us
I20260809 07:33:40.029803 31170 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:40.030490 31170 raft_consensus.cc:595] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.030884 31170 raft_consensus.cc:603] T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260809 07:33:40.035197 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 78582.9us
I20260809 07:33:40.033601 2866 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:40.026000 2872 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.040864 1856 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.039592 2866 raft_consensus.cc:482] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:40.043792 2866 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:40.044952 2891 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:40.045614 2892 consensus_queue.cc:1046] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:40.045953 30853 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.046279 2677 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:40.047868 30852 leader_election.cc:302] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.048482 1856 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.050179 31170 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c1ac5b8f9784e3b9c028fa7f7bdf1ea"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:40.050858 31170 raft_consensus.cc:595] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.051183 31170 raft_consensus.cc:603] T 8c1ac5b8f9784e3b9c028fa7f7bdf1ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:40.046862 2891 raft_consensus.cc:3032] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:40.061522 2891 raft_consensus.cc:729] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:40.058362 30853 leader_election.cc:302] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.049606 2738 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:40.059850 2835 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:40.060171 2689 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.056731 2673 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:40.073994 2483 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.072232 2541 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:40.071904 2689 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.072815 2714 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:40.082152 2714 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.083227 2689 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.084084 2714 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.068782 2738 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.078807 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.071904 2427 raft_consensus.cc:2726] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:40.081094 2541 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.085332 31172 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.088871 31170 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:40.088863 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.085985 31120 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.089639 31170 raft_consensus.cc:595] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.104447 31170 raft_consensus.cc:603] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:40.062490 2891 consensus_queue.cc:260] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.115948 2738 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.117479 30909 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:40.088673 2834 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.119036 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.089314 2673 raft_consensus.cc:686] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:40.118119 30909 raft_consensus.cc:595] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.104658 2561 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:40.127892 30909 raft_consensus.cc:603] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:40.140839 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
W20260809 07:33:40.145363 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 79539.1us
I20260809 07:33:40.130584 2891 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.131757 2669 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260809 07:33:40.168241 30852 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.176612 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.177317 30898 raft_consensus.cc:2454] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:40.174760 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.160704 2661 raft_consensus.cc:2781] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:40.200143 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.185742 31165 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:40.202683 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.203123 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.226225 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.224833 30906 consensus_queue.cc:260] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.179046 2948 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:40.223811 31165 raft_consensus.cc:595] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.233072 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.234756 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
W20260809 07:33:40.191965 3006 log.cc:924] Time spent T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.064s user 0.000s sys 0.001s
I20260809 07:33:40.236709 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.239405 30890 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
I20260809 07:33:40.236788 31170 raft_consensus.cc:2454] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:40.241364 30891 raft_consensus.cc:1226] T aeccd39240c34b5fabec551b21206a7f P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:40.202054 2665 raft_consensus.cc:3032] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:40.185567 2879 raft_consensus.cc:3032] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Stepping down as leader of term 5
I20260809 07:33:40.198681 2541 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.244534 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.245692 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.248172 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.252717 31172 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:40.236497 2517 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:40.255702 30911 raft_consensus.cc:1226] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 10. Current term is 11. Ops: []
I20260809 07:33:40.241155 2661 raft_consensus.cc:686] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:40.257270 30906 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.257714 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.258275 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:40.256237 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 88191.8us
I20260809 07:33:40.195324 2866 raft_consensus.cc:504] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.208688 2910 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260809 07:33:40.261821 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.244467 2665 raft_consensus.cc:729] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:40.272743 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.273386 30890 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.263546 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.247423 2879 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:40.265393 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.232110 2561 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260809 07:33:40.257869 2517 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:40.257591 2689 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:40.263487 2691 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.276451 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.287418 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.278668 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.276682 31172 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:40.204140 2673 consensus_queue.cc:237] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.278108 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.290534 30891 raft_consensus.cc:3037] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.288975 2691 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:40.297112 2691 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.286816 29021 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260809 07:33:40.288535 31165 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:40.260638 2955 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.281147 2517 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.304943 30852 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.318740 31170 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.322090 31120 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:40.331022 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 43968.3us
I20260809 07:33:40.332499 30852 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.333557 2679 raft_consensus.cc:2726] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260809 07:33:40.334399 3010 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.163s user 0.001s sys 0.000s
I20260809 07:33:40.338729 2745 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.339463 2745 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.340764 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.341512 30911 raft_consensus.cc:3037] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.343058 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.341589 2745 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:40.338536 2575 raft_consensus.cc:2781] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:40.344818 2575 raft_consensus.cc:686] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:40.345376 2866 leader_election.cc:288] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.324606 29021 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 07:33:40.344197 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.356393 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.357141 2711 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.338692 2665 consensus_queue.cc:260] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.358368 2665 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.358144 2711 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.360366 2711 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.362536 2897 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:40.362691 30852 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.338544 2945 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:40.325716 2926 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260809 07:33:40.375582 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.377655 2897 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:40.318039 2689 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.362552 2835 raft_consensus.cc:2726] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:40.387311 2945 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.389547 2945 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:40.389833 3022 log.cc:924] Time spent T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.066s user 0.001s sys 0.001s
I20260809 07:33:40.391424 2689 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.394392 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.395812 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.396972 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.398679 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.399338 30899 raft_consensus.cc:2454] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:40.400097 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.401229 2879 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.402285 2879 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:40.411633 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.416357 30906 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:40.418509 30891 raft_consensus.cc:2454] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:40.419265 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.420768 30898 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:40.345685 2575 consensus_queue.cc:237] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:40.426748 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 64159.6us
I20260809 07:33:40.429603 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.430871 1621 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260809 07:33:40.347177 2661 consensus_queue.cc:237] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.435520 1621 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.435876 1621 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:40.436151 1621 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.382457 2872 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:40.423161 2659 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260809 07:33:40.440336 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:40.452724 3020 log.cc:924] Time spent T 87ac258020ac4e29b08523647d3ab361 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260809 07:33:40.436636 2665 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:40.453471 2665 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.455386 2665 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.458671 2955 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:40.465619 1621 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.469061 1621 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:40.470409 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.473886 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:40.476310 30852 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:40.477623 2427 consensus_queue.cc:1035] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:40.479875 2897 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:40.476150 2835 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.483168 2835 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:40.483582 2835 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.485818 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:40.485601 30890 raft_consensus.cc:2454] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:40.473877 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.488541 30899 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.489086 31057 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.490085 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.491688 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.492956 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.493502 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.494750 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.495453 30906 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.496649 2836 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.498090 1621 consensus_queue.cc:1046] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:40.498296 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.498709 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.500383 2745 raft_consensus.cc:3032] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:40.500810 2745 raft_consensus.cc:729] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:40.501703 2745 consensus_queue.cc:260] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.500378 2427 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:40.501698 2835 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.502935 2745 raft_consensus.cc:3037] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.504050 2835 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.504618 2427 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:40.505061 2427 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.497256 2836 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.463043 2872 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.508086 2836 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.509434 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.510623 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.511938 31046 raft_consensus.cc:1226] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.511693 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.512380 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d4e67ddf074e90a2d36bc4e649b5c1" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.512630 30899 raft_consensus.cc:2354] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:40.513164 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.522189 31057 raft_consensus.cc:2396] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260809 07:33:40.524943 2836 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.525468 2836 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.526928 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.527550 31046 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.530011 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.530609 31051 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:40.545549 30904 raft_consensus.cc:2396] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260809 07:33:40.533769 31059 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:40.531814 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.556852 31046 raft_consensus.cc:2379] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:40.555627 30911 raft_consensus.cc:2454] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.556355 2872 raft_consensus.cc:482] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:40.558235 30742 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40564 (request call id 150) took 11335 ms (client timeout 15000 ms). Trace:
I20260809 07:33:40.558908 31120 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.559445 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.560031 31057 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.560381 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.560048 2872 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.561383 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.562618 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.563268 31051 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.509114 2691 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.579488 30891 raft_consensus.cc:1226] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.564570 30904 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.598466 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.599735 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.600544 30899 raft_consensus.cc:2354] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:40.579777 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.578481 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.603217 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3163efce7d454fb409a007f6ee93ef" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.604725 30906 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.579144 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.560763 2836 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.607028 31120 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.579165 2872 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.625730 31055 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.578148 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335fccb907184174b4a6428ce262e7dc" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.598040 2745 raft_consensus.cc:2781] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:40.629057 31046 raft_consensus.cc:2454] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:40.629526 31057 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:40.544313 2517 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.635068 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.636270 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.637352 31057 raft_consensus.cc:3037] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.640468 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.641801 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.642987 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.644464 31057 raft_consensus.cc:2454] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.646590 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.647533 31057 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:40.644515 31059 raft_consensus.cc:1261] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:40.649210 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.650111 31046 raft_consensus.cc:2396] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260809 07:33:40.653357 2691 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:40.661198 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.661949 31046 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.648067 31049 raft_consensus.cc:1261] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:40.665314 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:40.558652 30742 rpcz_store.cc:269] 0809 07:33:29.220499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:29.239151 (+ 18652us) service_pool.cc:227] Handling call
0809 07:33:40.522791 (+11283640us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:38.253279 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:38.253804 (+ 525us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:38.253844 (+ 40us) write_op.cc:435] Acquired schema lock
0809 07:33:38.253856 (+ 12us) tablet.cc:660] Decoding operations
0809 07:33:38.575369 (+321513us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:38.606734 (+ 31365us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:38.612974 (+ 6240us) tablet.cc:683] Acquiring locks for 11 operations
0809 07:33:38.870064 (+257090us) tablet.cc:699] Row locks acquired
0809 07:33:38.870100 (+ 36us) write_op.cc:260] PREPARE: finished
0809 07:33:38.912547 (+ 42447us) write_op.cc:270] Start()
0809 07:33:38.930041 (+ 17494us) write_op.cc:276] Timestamp: P: 1786260818884924 usec, L: 0
0809 07:33:38.930079 (+ 38us) op_driver.cc:331] REPLICATION: starting
0809 07:33:39.024497 (+ 94418us) log.cc:842] Serialized 4384 byte log entry
0809 07:33:39.208973 (+184476us) op_driver.cc:444] REPLICATION: finished
0809 07:33:39.224398 (+ 15425us) write_op.cc:301] APPLY: starting
0809 07:33:39.224502 (+ 104us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:39.224960 (+ 458us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:39.225133 (+ 173us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:39.251296 (+ 26163us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:39.251456 (+ 160us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0809 07:33:39.251485 (+ 29us) write_op.cc:312] APPLY: finished
0809 07:33:39.251682 (+ 197us) log.cc:842] Serialized 82 byte log entry
0809 07:33:39.251831 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:39.251981 (+ 150us) write_op.cc:454] Released schema lock
0809 07:33:39.252029 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:39.252289 (+ 260us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:39.252318 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":116489,"reported_tablets":57,"child_traces":[["op",{"apply.queue_time_us":14921,"apply.run_cpu_time_us":3635,"apply.run_wall_time_us":28360,"num_ops":11,"prepare.queue_time_us":827350,"prepare.run_cpu_time_us":8656,"prepare.run_wall_time_us":993416,"raft.queue_time_us":182951,"raft.run_cpu_time_us":27160,"raft.run_wall_time_us":302373,"replication_time_us":275782,"thread_start_us":740137,"threads_started":5,"wal-append.queue_time_us":164626}]]}
I20260809 07:33:40.682066 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.667268 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.768229 31054 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:40.776834 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6264779cf74c14bc8c18901dfe6a09" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.776391 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.780046 31051 raft_consensus.cc:2379] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:40.781152 2517 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:40.782485 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.783658 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ed1309ae2b34b6085b236210555d243" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.784660 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.785199 31058 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.796092 30745 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 31) took 2102 ms. Trace:
I20260809 07:33:40.741343 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:40.797624 30890 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:40.799814 31054 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.800460 31055 raft_consensus.cc:2379] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:40.801868 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.663570 2745 raft_consensus.cc:686] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:40.666651 2575 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:40.802516 31059 raft_consensus.cc:2454] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:40.804067 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.807205 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43645669985b45a0b7e6d18147b28424" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.808425 31054 raft_consensus.cc:1261] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:40.811517 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51e40c75108c4a86a26730860edee159" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.812744 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:40.817206 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.818674 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.822738 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.796499 30745 rpcz_store.cc:276] 0809 07:33:38.693912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:38.699299 (+ 5387us) service_pool.cc:227] Handling call
0809 07:33:40.796006 (+2096707us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 07:33:40.080343 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 07:33:40.080994 (+ 651us) write_op.cc:432] Acquiring schema lock in shared mode
0809 07:33:40.081052 (+ 58us) write_op.cc:435] Acquired schema lock
0809 07:33:40.081088 (+ 36us) tablet.cc:660] Decoding operations
0809 07:33:40.081525 (+ 437us) write_op.cc:620] Acquiring the partition lock for write op
0809 07:33:40.081609 (+ 84us) write_op.cc:641] Partition lock acquired for write op
0809 07:33:40.081650 (+ 41us) tablet.cc:683] Acquiring locks for 1 operations
0809 07:33:40.081931 (+ 281us) tablet.cc:699] Row locks acquired
0809 07:33:40.081965 (+ 34us) write_op.cc:260] PREPARE: finished
0809 07:33:40.082146 (+ 181us) write_op.cc:270] Start()
0809 07:33:40.082279 (+ 133us) write_op.cc:276] Timestamp: P: 1786260820082077 usec, L: 0
0809 07:33:40.085056 (+ 2777us) op_driver.cc:331] REPLICATION: starting
0809 07:33:40.125198 (+ 40142us) log.cc:842] Serialized 528 byte log entry
0809 07:33:40.347515 (+222317us) op_driver.cc:444] REPLICATION: finished
0809 07:33:40.667822 (+320307us) write_op.cc:301] APPLY: starting
0809 07:33:40.667935 (+ 113us) tablet.cc:1365] starting BulkCheckPresence
0809 07:33:40.668068 (+ 133us) tablet.cc:1368] finished BulkCheckPresence
0809 07:33:40.668189 (+ 121us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 07:33:40.668545 (+ 356us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 07:33:40.668785 (+ 240us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 07:33:40.668851 (+ 66us) write_op.cc:312] APPLY: finished
0809 07:33:40.747587 (+ 78736us) log.cc:842] Serialized 22 byte log entry
0809 07:33:40.747769 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:40.747856 (+ 87us) write_op.cc:454] Released schema lock
0809 07:33:40.747907 (+ 51us) write_op.cc:341] FINISH: Updating metrics
0809 07:33:40.748170 (+ 263us) write_op.cc:489] Releasing partition, row and schema locks
0809 07:33:40.748200 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":182606,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":313281,"apply.run_cpu_time_us":2292,"apply.run_wall_time_us":80656,"num_ops":1,"prepare.queue_time_us":176691,"prepare.run_cpu_time_us":3367,"prepare.run_wall_time_us":65915,"replication_time_us":238886,"thread_start_us":270310,"threads_started":2}]]}
I20260809 07:33:40.824057 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.825037 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.825814 31055 raft_consensus.cc:1261] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260809 07:33:40.826072 31054 raft_consensus.cc:2454] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:40.829553 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.825364 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.833935 2926 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.834483 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet 335fccb907184174b4a6428ce262e7dc with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:43208
I20260809 07:33:40.838469 2926 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.828781 2427 raft_consensus.cc:504] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:40.835150 30904 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 07:33:40.835824 31005 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:40.840503 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:40.840651 2926 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:40.839466 31045 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.833487 30853 leader_election.cc:302] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:40.841583 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.817042 2575 raft_consensus.cc:686] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:40.830871 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.850803 31005 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:40.847080 2879 raft_consensus.cc:2781] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.851524 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.844532 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.853058 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.854018 30891 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.855507 30900 raft_consensus.cc:1226] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:40.853767 31048 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 7.
I20260809 07:33:40.853219 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.855949 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:40.856460 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd4a4091f24d51a4fae9e8d4ebca9e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.852948 31055 raft_consensus.cc:2454] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 6.
I20260809 07:33:40.853246 2879 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:40.859691 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.843888 2661 raft_consensus.cc:2781] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:40.844022 2836 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:40.871482 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.870307 31055 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.858250 2672 raft_consensus.cc:2781] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:40.873806 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.882009 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.865586 2879 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:40.896746 31177 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:40.854708 2926 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
W20260809 07:33:40.899614 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48907.1us
I20260809 07:33:40.901060 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:40.890146 30890 raft_consensus.cc:1226] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.891261 2427 leader_election.cc:288] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:40.891528 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:40.892231 30724 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 3bbfdffc27ac46a0a18582f85cd4a00a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2c9de56c3fa3487c9dac990675c34be5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:40.867107 2955 consensus_queue.cc:1046] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260809 07:33:40.909991 31006 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:40.884402 2661 raft_consensus.cc:686] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:40.900498 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:40.912479 31049 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 7.
I20260809 07:33:40.915277 31172 raft_consensus.cc:1226] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.915618 31048 raft_consensus.cc:1261] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:40.878531 2672 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:40.912441 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.920338 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.926226 31182 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:40.927266 31182 raft_consensus.cc:1226] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.923370 31169 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 5. Current term is 6. Ops: []
W20260809 07:33:40.958909 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 58504.5us
I20260809 07:33:40.961455 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.961603 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.963447 31049 raft_consensus.cc:1261] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 07:33:40.965153 31181 raft_consensus.cc:2354] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:40.968649 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.918758 2910 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:40.975946 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bd1fdaac4346aa9f2f9be0f904b408" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:40.976786 31006 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:40.978088 31048 raft_consensus.cc:1261] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:40.985954 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:40.990106 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.957052 2672 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:40.960268 2836 raft_consensus.cc:2781] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:40.993988 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb9e791b482439b936398015bc42aff" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:40.995409 2669 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:40.995957 2569 consensus_queue.cc:1046] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260809 07:33:40.960594 30724 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 77313.1us
I20260809 07:33:41.005816 31057 raft_consensus.cc:1226] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:41.006152 31058 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:41.007740 31045 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:40.886165 2745 consensus_queue.cc:237] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.024973 31164 raft_consensus.cc:1226] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:40.988392 2738 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 07:33:41.044015 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 148713us
I20260809 07:33:41.045907 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656029f31f2846908145dfacac4abdf2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:41.049690 31059 raft_consensus.cc:2379] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:41.023335 2669 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:40.989550 2910 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.015944 2836 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:41.052358 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:41.091218 31172 raft_consensus.cc:1261] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:41.092520 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.094599 31164 raft_consensus.cc:2454] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:41.017733 2872 consensus_queue.cc:1046] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260809 07:33:41.074222 2669 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:40.976886 2575 consensus_queue.cc:237] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.127290 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b57d0ed4ced43cfbe36695b39b05897" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:41.128011 31058 raft_consensus.cc:3037] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.130990 31059 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260809 07:33:41.098145 2836 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.133134 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.133260 31059 raft_consensus.cc:1226] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260809 07:33:41.140276 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:41.139081 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:41.143587 31059 raft_consensus.cc:2379] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:41.148150 31058 raft_consensus.cc:2454] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:41.150192 31058 raft_consensus.cc:1226] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:41.046947 2661 consensus_queue.cc:237] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:41.164127 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 150632us
W20260809 07:33:41.171686 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 194468us
I20260809 07:33:41.175989 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.177316 31005 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.183374 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8db10c4bb04e86a15bcbff5ffed3d5" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:41.198073 31059 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:41.215916 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 54353.6us
I20260809 07:33:41.218304 3076 raft_consensus.cc:3032] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 LEADER]: Stepping down as leader of term 10
I20260809 07:33:41.241496 1621 raft_consensus.cc:3032] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:41.242079 1621 raft_consensus.cc:729] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:41.242991 1621 consensus_queue.cc:260] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.278247 31055 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:41.240370 3076 raft_consensus.cc:729] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.273887 2910 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:41.283947 3083 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:41.294919 3083 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
W20260809 07:33:41.325732 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 148515us
I20260809 07:33:41.282241 3085 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.331185 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:41.331943 31055 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9965e21374d26b0ac2014c9bafdff in term 9.
I20260809 07:33:41.327696 31005 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.337699 31005 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:41.278769 3075 consensus_queue.cc:1046] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260809 07:33:41.327665 3097 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:41.352149 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.351049 3076 consensus_queue.cc:260] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.345952 3085 raft_consensus.cc:2726] T c1ccd6862a094ecba07b495d4c772721 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.375236 31005 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:41.355362 3108 raft_consensus.cc:2781] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 07:33:41.350950 3083 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:41.377421 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 140446us
I20260809 07:33:41.389884 31005 leader_election.cc:302] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.379511 3076 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:41.359221 3097 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.401106 31059 raft_consensus.cc:2454] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:41.401535 3097 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.403045 3103 raft_consensus.cc:2726] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260809 07:33:41.400007 31051 raft_consensus.cc:1261] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260809 07:33:41.403749 3103 raft_consensus.cc:2781] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:41.379730 3108 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:41.418354 30724 catalog_manager.cc:4841] TS 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749): tablet 335fccb907184174b4a6428ce262e7dc (table test-workload [id=0d3996321a004d1e858c58dfe86c0dc4]) successfully deleted
I20260809 07:33:41.420482 3075 raft_consensus.cc:3032] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:41.420856 3075 raft_consensus.cc:729] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.421636 3075 consensus_queue.cc:260] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.422794 2872 raft_consensus.cc:3032] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:41.423200 2872 raft_consensus.cc:729] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.423933 2872 consensus_queue.cc:260] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.424937 2872 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.425639 2879 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.418645 31051 raft_consensus.cc:1261] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260809 07:33:41.435681 3046 log.cc:924] Time spent T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.585s user 0.007s sys 0.011s
I20260809 07:33:41.404531 2672 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:41.436795 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 224500us
I20260809 07:33:41.408882 3085 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.438627 31059 raft_consensus.cc:1226] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 11. Current term is 12. Ops: []
I20260809 07:33:41.440126 3103 raft_consensus.cc:686] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.416697 3108 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:41.443475 31054 raft_consensus.cc:2379] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:41.409076 3097 consensus_queue.cc:1046] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260809 07:33:41.445315 31048 raft_consensus.cc:1226] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:41.432334 2669 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.454422 31058 raft_consensus.cc:3032] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:41.454833 31058 raft_consensus.cc:729] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.455590 31058 consensus_queue.cc:260] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.456523 31058 raft_consensus.cc:3037] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.456905 2672 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.458029 31045 raft_consensus.cc:2354] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:41.457600 3103 consensus_queue.cc:237] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.454421 2659 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260809 07:33:41.454917 31055 raft_consensus.cc:1261] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 07:33:41.473317 31058 raft_consensus.cc:1261] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260809 07:33:41.473759 31058 pending_rounds.cc:76] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 3
W20260809 07:33:41.474972 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 85183us
I20260809 07:33:41.474903 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:41.476500 31005 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.477284 30911 raft_consensus.cc:2396] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28b9965e21374d26b0ac2014c9bafdff for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260809 07:33:41.473984 31058 pending_rounds.cc:97] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260809 07:33:41.485160 31058 raft_consensus.cc:2866] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:41.477770 31005 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.481967 31172 raft_consensus.cc:1261] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 07:33:41.488745 30852 leader_election.cc:302] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.490440 31005 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.494073 31005 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.455605 2669 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.501401 31005 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.502972 31005 leader_election.cc:302] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.478279 2834 consensus_queue.cc:1046] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260809 07:33:41.506732 31164 raft_consensus.cc:1261] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:41.509330 31005 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.510656 31005 leader_election.cc:398] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:41.510949 31005 leader_election.cc:401] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260809 07:33:41.562729 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52207.7us
I20260809 07:33:41.564412 31005 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.491506 2659 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:41.486989 3112 raft_consensus.cc:2726] T ee4aa83021af4237806656803bff3e10 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.491655 3097 raft_consensus.cc:2726] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.497907 3075 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.504688 3076 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.507704 3098 raft_consensus.cc:2726] T 4930be41b06547ba93e04e5d1385cb3c P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:41.504972 3085 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260809 07:33:41.653947 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 265172us
I20260809 07:33:41.700778 31006 leader_election.cc:398] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Vote denied by peer 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107) with higher term. Message: Invalid argument: T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
W20260809 07:33:41.567412 3109 log.cc:924] Time spent T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.123s user 0.000s sys 0.001s
I20260809 07:33:41.578152 2659 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
W20260809 07:33:41.644856 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 80520us
I20260809 07:33:41.582721 2679 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260809 07:33:41.642249 2673 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.706571 31006 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.566522 2672 raft_consensus.cc:2726] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.556313 2669 raft_consensus.cc:2726] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.569336 3089 raft_consensus.cc:2726] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
W20260809 07:33:41.659991 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 171327us
W20260809 07:33:41.652405 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 150285us
W20260809 07:33:41.680554 2836 consensus_meta.cc:287] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a: Time spent flushing consensus metadata: real 0.537s user 0.008s sys 0.003s
I20260809 07:33:41.711364 31164 raft_consensus.cc:1261] T 401ec3203e7b44faae0c513339cc7317 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 07:33:41.695228 3112 raft_consensus.cc:2726] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.664965 2879 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.714155 31005 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.717437 31006 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:41.707351 3124 log.cc:924] Time spent T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.068s user 0.001s sys 0.004s
I20260809 07:33:41.725912 31005 leader_election.cc:398] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:41.726234 31005 leader_election.cc:401] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260809 07:33:41.724174 3117 raft_consensus.cc:2726] T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.728993 2672 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:41.731791 2672 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:41.732198 2672 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.729075 30852 leader_election.cc:302] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.724725 3112 raft_consensus.cc:2726] T b72ff16aedce4676b889395ca4d8909d P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.736800 3103 raft_consensus.cc:2726] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) with higher term. Message: Invalid argument: T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8.
I20260809 07:33:41.728183 3134 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:41.738757 3134 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.740851 3134 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.736279 3116 consensus_queue.cc:1035] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260809 07:33:41.746838 30852 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.746941 3136 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:41.746340 3128 raft_consensus.cc:2726] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.747676 2661 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:41.748140 30852 leader_election.cc:302] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.749755 3078 raft_consensus.cc:3032] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:41.750176 3078 raft_consensus.cc:729] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.751068 2835 raft_consensus.cc:2726] T 5ed1309ae2b34b6085b236210555d243 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.751111 3078 consensus_queue.cc:260] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.753140 3078 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:41.754211 1621 raft_consensus.cc:3037] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:41.754642 30852 leader_election.cc:302] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.757288 30852 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.758976 3123 raft_consensus.cc:2726] T 51e40c75108c4a86a26730860edee159 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.762084 31005 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.770773 30899 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:41.772680 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:41.781209 31005 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.783708 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.715652 2836 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.788530 31174 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:41.789732 31005 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.791193 30852 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.795668 30852 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.795801 31005 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.796877 30896 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:41.802788 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e26e3128404ae8824fab16033fc0ea" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:41.788532 3127 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.081s user 0.003s sys 0.000s
I20260809 07:33:41.785966 2679 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:41.785416 2872 raft_consensus.cc:2726] T 976fdb751bc0440a92f97b6c95314270 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.807919 31057 raft_consensus.cc:1226] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:41.809438 31054 raft_consensus.cc:2454] T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
W20260809 07:33:41.788810 3114 log.cc:924] Time spent T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.193s user 0.000s sys 0.015s
I20260809 07:33:41.806458 3117 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.811156 3113 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.790536 3112 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:41.811677 3102 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.812034 3112 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:41.790570 3061 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.689s user 0.003s sys 0.010s
I20260809 07:33:41.734128 2879 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260809 07:33:41.817216 2659 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
I20260809 07:33:41.818125 3108 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.818886 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:41.820493 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:41.822052 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:41.823648 3120 consensus_queue.cc:1035] T dd3163efce7d454fb409a007f6ee93ef P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:41.825357 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff" is_pre_election: true
I20260809 07:33:41.826166 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:41.811398 3103 raft_consensus.cc:2726] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.793895 3088 raft_consensus.cc:2726] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.826071 32197 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:41.797789 3097 raft_consensus.cc:2726] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.834447 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.837360 31181 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:41.840104 2897 raft_consensus.cc:3032] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:41.840514 2897 raft_consensus.cc:729] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:41.841282 2897 consensus_queue.cc:260] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.804592 2835 consensus_queue.cc:1035] T 87ac258020ac4e29b08523647d3ab361 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 07:33:41.845026 2672 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.847036 2672 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.826856 2679 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.850438 2679 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.857045 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.858223 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.859289 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:41.861660 3088 consensus_queue.cc:1035] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:41.865119 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:41.853541 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:41.871917 31006 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.876924 31165 raft_consensus.cc:1226] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:41.878860 31175 raft_consensus.cc:1226] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer a720817fc113446a8ed55e61523b280a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260809 07:33:41.868381 2738 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s user 0.013s sys 0.019s
I20260809 07:33:41.880759 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:41.884825 31181 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:41.882326 30852 leader_election.cc:302] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.880434 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:41.883090 31005 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.883915 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:41.876065 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:41.887818 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:41.886636 31006 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.893182 31006 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:41.878185 2897 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:41.889797 3085 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260809 07:33:41.894085 3117 consensus_queue.cc:1046] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 07:33:41.895161 3117 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 07:33:41.902786 2836 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.890441 3076 raft_consensus.cc:2726] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 07:33:41.886327 2835 consensus_queue.cc:1035] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260809 07:33:41.883316 3097 raft_consensus.cc:2726] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:41.886734 30849 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:41.887137 2738 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s user 0.014s sys 0.019s
I20260809 07:33:41.919852 2738 raft_consensus.cc:3032] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:41.920212 2738 raft_consensus.cc:729] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:41.920944 2738 consensus_queue.cc:260] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:41.923105 2548 raft_consensus.cc:2726] T 0cb9e791b482439b936398015bc42aff P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.923576 3108 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 11 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.924423 30852 leader_election.cc:302] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.924798 30853 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:41.925875 3073 raft_consensus.cc:2726] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.926496 3073 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:41.926836 3073 raft_consensus.cc:482] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:41.927141 3073 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:41.922106 2738 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
W20260809 07:33:41.939073 30849 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.68:39299, user_credentials={real_user=slave}} blocked reactor thread for 52417.5us
I20260809 07:33:41.962165 3112 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:41.949856 2835 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:41.972705 2835 raft_consensus.cc:482] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:41.973104 3136 consensus_queue.cc:1046] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 07:33:41.975816 31051 raft_consensus.cc:1226] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:41.975405 30741 catalog_manager.cc:5495] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:41.981904 30852 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.990427 2673 raft_consensus.cc:2726] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260809 07:33:41.982852 2835 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:41.983093 2659 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:41.992139 30852 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:41.984026 3088 raft_consensus.cc:3032] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:42.004832 30852 leader_election.cc:302] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:42.008044 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 116568us
I20260809 07:33:42.009286 31005 leader_election.cc:302] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.001297 2548 raft_consensus.cc:2726] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:41.999850 3088 raft_consensus.cc:729] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.016464 3120 raft_consensus.cc:2726] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:42.017023 3088 consensus_queue.cc:260] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.017639 3107 consensus_queue.cc:1046] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 07:33:42.022068 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:42.022921 2661 consensus_queue.cc:1035] T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:42.006995 30741 catalog_manager.cc:5495] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:42.010423 2948 raft_consensus.cc:2781] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
W20260809 07:33:42.038826 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 162345us
W20260809 07:33:42.047281 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 154166us
I20260809 07:33:42.003162 2659 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.004629 3113 consensus_queue.cc:1046] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260809 07:33:42.048544 31006 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.047703 31120 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:42.059146 3137 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.112s user 0.001s sys 0.000s
I20260809 07:33:42.060808 3113 raft_consensus.cc:3032] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:42.061460 3113 raft_consensus.cc:729] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.062469 2948 raft_consensus.cc:686] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.063879 2948 consensus_queue.cc:237] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.070008 30741 catalog_manager.cc:5495] T ca210f0f3a954ffb8a42764c24b02830 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 6, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:42.079617 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb2fb324ee14e2084a95783b59cab00" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:42.081163 31175 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260809 07:33:42.081764 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 46073.4us
I20260809 07:33:42.085939 2910 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:42.086257 3113 consensus_queue.cc:260] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.087014 2910 raft_consensus.cc:686] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:42.081173 30741 catalog_manager.cc:5495] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 5 to 6, leader changed from 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65) to a720817fc113446a8ed55e61523b280a (127.28.87.66). New cstate: current_term: 6 leader_uuid: "a720817fc113446a8ed55e61523b280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } health_report { overall_health: HEALTHY } } }
I20260809 07:33:42.088765 30741 catalog_manager.cc:5495] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:42.088951 2910 consensus_queue.cc:237] T 9dba78a1408b4bd7a6f0c247b1176f0b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.090432 30741 catalog_manager.cc:5495] T 24dd4a4091f24d51a4fae9e8d4ebca9e P a720817fc113446a8ed55e61523b280a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "12107e38eb6d424abc4f45f16a9a1560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:42.083148 31006 leader_election.cc:302] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.097741 2659 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.090718 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:42.099537 30852 leader_election.cc:302] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.105818 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.076905 3088 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.092592 2673 raft_consensus.cc:3032] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 LEADER]: Stepping down as leader of term 11
I20260809 07:33:42.095104 3098 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260809 07:33:42.118556 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 70934.4us
I20260809 07:33:42.125300 3059 consensus_queue.cc:1046] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:42.125355 3120 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:42.127288 3059 raft_consensus.cc:3032] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:42.127410 31006 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.130841 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.121141 30852 leader_election.cc:398] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:42.129020 3112 raft_consensus.cc:2726] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.146693 2955 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:42.147231 2955 raft_consensus.cc:482] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.136627 2679 raft_consensus.cc:482] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:42.147670 2955 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:42.148008 2679 raft_consensus.cc:504] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.157603 2679 leader_election.cc:288] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.158241 3149 raft_consensus.cc:482] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.158888 3149 raft_consensus.cc:504] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.160873 3149 leader_election.cc:288] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.158198 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.164426 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.111626 3085 raft_consensus.cc:2726] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.119045 2659 raft_consensus.cc:3037] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.104801 3116 consensus_queue.cc:1035] T 5e48f193f20844a4ac619300cfaac1bd P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260809 07:33:42.168648 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeccd39240c34b5fabec551b21206a7f" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:42.119503 2673 raft_consensus.cc:729] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.127933 3059 raft_consensus.cc:729] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:42.169660 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.108196 2427 consensus_queue.cc:1035] T 401ec3203e7b44faae0c513339cc7317 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260809 07:33:42.178993 2691 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:42.179477 2691 raft_consensus.cc:482] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.179699 31120 leader_election.cc:302] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.180119 2691 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.179389 3059 consensus_queue.cc:260] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.180751 2575 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.129426 2897 consensus_queue.cc:1046] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260809 07:33:42.181432 3059 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.182086 3102 raft_consensus.cc:3032] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:42.182498 3102 raft_consensus.cc:729] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.184005 2673 consensus_queue.cc:260] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.179406 2955 raft_consensus.cc:504] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.185568 3073 raft_consensus.cc:504] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.187623 31120 leader_election.cc:302] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.191326 31005 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.178306 3113 raft_consensus.cc:3037] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.213115 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.214454 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.219386 31048 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:42.185220 2673 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Advancing to term 12
I20260809 07:33:42.216588 30899 raft_consensus.cc:1261] T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:42.215874 31051 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:42.220353 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca210f0f3a954ffb8a42764c24b02830" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.217479 31058 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:42.227277 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.229352 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.230005 31045 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:42.232586 31046 raft_consensus.cc:1226] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 07:33:42.207407 3085 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:42.237764 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.238281 31051 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:42.257377 31057 pending_rounds.cc:76] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Aborting all ops after (but not including) 4
I20260809 07:33:42.257696 31057 pending_rounds.cc:97] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260809 07:33:42.257973 31057 raft_consensus.cc:2866] T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 07:33:42.229394 3128 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
W20260809 07:33:42.194517 2897 consensus_queue.cc:1159] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "12107e38eb6d424abc4f45f16a9a1560" responder_term: 11 status { last_received { term: 11 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 11 index: 7 } }
I20260809 07:33:42.210060 2891 consensus_queue.cc:1046] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260809 07:33:42.251322 2910 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:42.283654 3104 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:42.229214 3120 consensus_queue.cc:1035] T 656029f31f2846908145dfacac4abdf2 P a720817fc113446a8ed55e61523b280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260809 07:33:42.293565 3073 leader_election.cc:288] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.216948 3102 consensus_queue.cc:260] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.283610 2891 raft_consensus.cc:3032] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:42.289359 3128 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.342631 3128 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.355404 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:42.356292 31174 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:42.285560 2910 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.370724 2955 leader_election.cc:288] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:42.371008 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
W20260809 07:33:42.371788 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 144601us
I20260809 07:33:42.291733 2904 raft_consensus.cc:2726] T f4d4e67ddf074e90a2d36bc4e649b5c1 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:42.373643 31169 raft_consensus.cc:1261] T 5b57d0ed4ced43cfbe36695b39b05897 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:42.378690 31120 leader_election.cc:302] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.388935 2835 raft_consensus.cc:504] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:42.348861 2891 raft_consensus.cc:729] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.303263 3104 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.391325 2575 raft_consensus.cc:2726] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.392357 3069 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:42.373102 2910 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:42.426589 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.427487 31051 raft_consensus.cc:2379] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:42.409346 2904 raft_consensus.cc:2726] T dd3163efce7d454fb409a007f6ee93ef P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.432682 2691 raft_consensus.cc:504] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.414482 3102 raft_consensus.cc:3037] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
W20260809 07:33:42.437572 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 58970.8us
I20260809 07:33:42.488829 31120 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.479125 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31954ed4d71d4a6796aa3c4bb2ded761" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:42.490163 31054 raft_consensus.cc:3037] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:42.484503 31174 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:42.495253 31120 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
W20260809 07:33:42.487106 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 55776.6us
I20260809 07:33:42.497886 1621 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:42.498265 1621 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:42.498600 1621 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.498888 30852 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.501116 30852 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:42.502056 31120 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.508049 31054 raft_consensus.cc:2454] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:42.516330 1621 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.518726 1621 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:42.519137 30852 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:42.500437 2575 raft_consensus.cc:2726] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.501600 3134 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:42.490427 3136 consensus_queue.cc:1035] T 5b57d0ed4ced43cfbe36695b39b05897 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260809 07:33:42.511500 2834 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:42.519307 2669 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:42.486061 2891 consensus_queue.cc:260] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:42.597334 3145 log.cc:924] Time spent T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
W20260809 07:33:42.595582 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 97299.3us
I20260809 07:33:42.579082 2904 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:42.627345 31120 leader_election.cc:302] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: 28b9965e21374d26b0ac2014c9bafdff, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.606261 2834 raft_consensus.cc:482] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:42.581094 3107 consensus_queue.cc:1046] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260809 07:33:42.636276 3088 consensus_meta.cc:287] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.523s user 0.010s sys 0.000s
W20260809 07:33:42.664770 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 145764us
I20260809 07:33:42.630632 3164 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:42.647121 2904 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:42.620591 2669 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.653198 2834 raft_consensus.cc:3037] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.656841 2575 raft_consensus.cc:2726] T 335fccb907184174b4a6428ce262e7dc P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.654783 3085 raft_consensus.cc:482] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.671965 3164 raft_consensus.cc:482] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:42.675774 30852 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.684554 31120 leader_election.cc:302] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.671187 3104 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.668632 3166 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:42.670928 2661 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:42.693105 30852 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:42.671964 2879 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:42.704808 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.708734 1621 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:42.709102 1621 raft_consensus.cc:482] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.709403 1621 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:42.677605 2427 raft_consensus.cc:3032] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
W20260809 07:33:42.693650 3160 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.199s user 0.001s sys 0.000s
I20260809 07:33:42.722765 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:42.723618 31051 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:42.683027 3164 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:42.729246 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.693744 2575 raft_consensus.cc:2726] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 07:33:42.694447 2679 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.729991 31058 raft_consensus.cc:2379] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:42.733500 1621 raft_consensus.cc:504] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.760526 2835 leader_election.cc:288] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107), 28b9965e21374d26b0ac2014c9bafdff (127.28.87.68:39299)
W20260809 07:33:42.761159 3172 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.157s user 0.001s sys 0.000s
I20260809 07:33:42.713034 2955 consensus_queue.cc:1035] T 43743864250e4199a5862ae02b5dbed7 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260809 07:33:42.763741 3167 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.166s user 0.000s sys 0.001s
W20260809 07:33:42.764284 3170 log.cc:924] Time spent T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.266s user 0.001s sys 0.000s
I20260809 07:33:42.712673 2879 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.767994 2879 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.768096 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28b9965e21374d26b0ac2014c9bafdff"
I20260809 07:33:42.768751 32197 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 9 FOLLOWER]: Advancing to term 10
W20260809 07:33:42.769971 3113 consensus_meta.cc:287] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.547s user 0.009s sys 0.000s
W20260809 07:33:42.772397 3059 consensus_meta.cc:287] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560: Time spent flushing consensus metadata: real 0.590s user 0.008s sys 0.001s
I20260809 07:33:42.681258 3085 raft_consensus.cc:504] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.774344 3085 leader_election.cc:288] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.722018 3166 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.776166 3166 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:42.776506 2659 consensus_meta.cc:287] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5: Time spent flushing consensus metadata: real 0.605s user 0.010s sys 0.000s
I20260809 07:33:42.776881 2659 raft_consensus.cc:2726] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.729000 3103 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:42.782035 3192 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:42.742038 2910 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.782637 3192 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.726984 3146 raft_consensus.cc:482] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.783735 3146 raft_consensus.cc:504] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.783347 31170 raft_consensus.cc:1226] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:42.786813 2891 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:42.788188 2910 leader_election.cc:288] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:42.763989 2955 consensus_queue.cc:1035] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:42.793323 3192 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.727985 2691 leader_election.cc:288] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:42.723564 3073 raft_consensus.cc:2781] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:42.799211 3073 raft_consensus.cc:686] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:42.800177 3146 leader_election.cc:288] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.800037 3073 consensus_queue.cc:237] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.782142 3103 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.785199 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.809900 31177 raft_consensus.cc:2379] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 6: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:42.807443 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.785933 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:42.806766 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.806288 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
W20260809 07:33:42.735080 3185 log.cc:924] Time spent T 656029f31f2846908145dfacac4abdf2 P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
I20260809 07:33:42.712934 2427 raft_consensus.cc:729] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
W20260809 07:33:42.813012 3182 log.cc:924] Time spent T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.210s user 0.008s sys 0.005s
I20260809 07:33:42.813282 2427 consensus_queue.cc:260] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.736912 2669 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.810503 31164 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:42.808136 31048 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:42.815068 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.816762 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:42.819386 31058 raft_consensus.cc:2379] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:42.817346 1621 leader_election.cc:288] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:42.809705 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.820212 31170 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 6.
W20260809 07:33:42.814395 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 101805us
I20260809 07:33:42.817504 31054 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:42.830679 31006 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:42.830750 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:42.831606 31006 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
W20260809 07:33:42.816368 3160 log.cc:924] Time spent T 66e26e3128404ae8824fab16033fc0ea P 2c9de56c3fa3487c9dac990675c34be5: Append to log took a long time: real 0.086s user 0.002s sys 0.001s
I20260809 07:33:42.812956 3103 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:42.726183 2661 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:42.798012 2834 raft_consensus.cc:504] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:42.834786 30724 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 3bbfdffc27ac46a0a18582f85cd4a00a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 2c9de56c3fa3487c9dac990675c34be5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } } }
I20260809 07:33:42.836917 2879 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:42.840073 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:42.846107 30890 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:42.836220 31120 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:42.826417 2427 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:42.849350 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:42.842758 31051 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:42.843648 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:42.844667 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a18497352404d2491d49a9241fe61bf" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:42.845381 2879 raft_consensus.cc:482] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:42.849947 30909 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 5.
I20260809 07:33:42.835071 3085 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:42.857710 3085 raft_consensus.cc:482] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:42.858144 3085 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:42.859762 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25fc40767244ba0873588714d88e8b6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:42.860918 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:42.847380 3116 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260809 07:33:42.857460 2948 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:42.863830 2948 raft_consensus.cc:482] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:42.864171 2948 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:42.854053 2879 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:42.741600 2904 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
W20260809 07:33:42.883219 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47060.6us
I20260809 07:33:42.874872 3205 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 07:33:42.885529 31174 raft_consensus.cc:1226] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:43.014245 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.013757 31181 raft_consensus.cc:1226] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:43.015355 31182 raft_consensus.cc:1261] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260809 07:33:43.020617 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.046490 31164 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
W20260809 07:33:42.997027 3204 log.cc:924] Time spent T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.099s user 0.001s sys 0.001s
I20260809 07:33:43.048331 30853 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.049759 3123 consensus_queue.cc:1046] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:42.987406 32197 raft_consensus.cc:2454] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:43.052345 3116 raft_consensus.cc:3032] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:43.051074 2835 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 5 FOLLOWER]: Advancing to term 6
I20260809 07:33:43.052699 3116 raft_consensus.cc:729] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.053488 3116 consensus_queue.cc:260] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.054800 30849 leader_election.cc:302] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28b9965e21374d26b0ac2014c9bafdff, 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
W20260809 07:33:42.936079 3170 log.cc:924] Time spent T 59bd1fdaac4346aa9f2f9be0f904b408 P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260809 07:33:43.055680 31120 leader_election.cc:302] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
W20260809 07:33:42.884814 3174 log.cc:924] Time spent T aeccd39240c34b5fabec551b21206a7f P a720817fc113446a8ed55e61523b280a: Append to log took a long time: real 0.186s user 0.007s sys 0.000s
I20260809 07:33:43.057839 1621 raft_consensus.cc:3037] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:43.063941 31120 leader_election.cc:302] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.065057 31120 leader_election.cc:302] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:42.936192 3188 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.072163 31120 leader_election.cc:302] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.078634 1621 raft_consensus.cc:2726] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:42.966251 3189 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.080680 31120 leader_election.cc:302] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.081593 1621 raft_consensus.cc:2726] T 43645669985b45a0b7e6d18147b28424 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 07:33:43.082331 1621 raft_consensus.cc:2781] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:43.082721 1621 raft_consensus.cc:482] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:42.935585 3150 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.083138 1621 raft_consensus.cc:3037] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.085393 3188 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.086462 3112 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.093494 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters: a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.066721 3205 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:43.062232 3102 raft_consensus.cc:2781] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:43.019735 3085 raft_consensus.cc:504] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:43.103749 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 61143.3us
I20260809 07:33:43.072922 2910 raft_consensus.cc:2781] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:43.073561 2691 raft_consensus.cc:2726] T ca210f0f3a954ffb8a42764c24b02830 P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:43.108538 2679 raft_consensus.cc:2726] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 07:33:43.112882 3166 consensus_queue.cc:1046] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260809 07:33:43.103125 3112 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.136206 31164 raft_consensus.cc:1226] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:43.100345 1621 raft_consensus.cc:504] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.083690 2659 consensus_queue.cc:1046] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260809 07:33:43.083690 3192 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.062875 2955 consensus_queue.cc:1046] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260809 07:33:43.142179 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:43.147970 3192 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.148466 2955 raft_consensus.cc:3032] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:43.148633 31051 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:43.148945 2955 raft_consensus.cc:729] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:43.116525 31120 leader_election.cc:302] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.143388 31005 leader_election.cc:302] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.113639 3123 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:43.149873 2955 consensus_queue.cc:260] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.150660 3123 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.144373 30890 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:43.145783 3112 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.151129 3123 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:43.151103 2955 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:43.110059 3116 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.153342 1621 leader_election.cc:288] T c1ccd6862a094ecba07b495d4c772721 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:43.093811 3136 consensus_queue.cc:1035] T cb8db10c4bb04e86a15bcbff5ffed3d5 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260809 07:33:43.154846 31120 leader_election.cc:302] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.155668 1621 raft_consensus.cc:2781] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 07:33:43.168756 3218 raft_consensus.cc:2726] T 1a18497352404d2491d49a9241fe61bf P a720817fc113446a8ed55e61523b280a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 07:33:43.169265 2691 raft_consensus.cc:2726] T 24dd4a4091f24d51a4fae9e8d4ebca9e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:43.170011 1621 raft_consensus.cc:482] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:43.170460 1621 raft_consensus.cc:3037] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:43.170955 31005 leader_election.cc:302] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.171644 3218 raft_consensus.cc:2726] T f25fc40767244ba0873588714d88e8b6 P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:43.168802 31120 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.174080 3120 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.175457 3120 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.177433 3120 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.180291 3166 raft_consensus.cc:3032] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 07:33:43.180694 3166 raft_consensus.cc:729] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.181497 3166 consensus_queue.cc:260] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.210615 3166 raft_consensus.cc:3037] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:43.183387 2659 raft_consensus.cc:3032] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:43.212580 2659 raft_consensus.cc:729] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.213470 2659 consensus_queue.cc:260] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.224362 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.142475 3164 raft_consensus.cc:504] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.227336 3164 leader_election.cc:288] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:43.228219 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 184482us
I20260809 07:33:43.122862 2948 raft_consensus.cc:504] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.093436 3150 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.118287 2910 raft_consensus.cc:482] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:43.191221 3085 leader_election.cc:288] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.191588 2691 raft_consensus.cc:2726] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:43.093202 3189 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.192710 3188 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.197490 2834 leader_election.cc:288] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.200013 2879 raft_consensus.cc:504] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.197917 31120 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.121866 2904 consensus_queue.cc:1035] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.111s
W20260809 07:33:43.245656 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 47833.6us
I20260809 07:33:43.203811 2835 raft_consensus.cc:2726] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 07:33:43.204137 1621 raft_consensus.cc:504] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:43.209656 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 73555.1us
I20260809 07:33:43.129869 3205 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.232424 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.250826 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.235886 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.235152 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.137162 3102 raft_consensus.cc:686] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 10 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.239775 31174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.241344 29021 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 07:33:43.252610 31182 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:43.252966 31181 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:43.254956 31006 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.255843 1621 leader_election.cc:288] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:43.238638 2910 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:43.250252 2691 raft_consensus.cc:2726] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 07:33:43.264222 3189 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.246407 31120 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560; no voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.268323 3150 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.268635 2948 leader_election.cc:288] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.268934 2835 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:43.269371 2835 raft_consensus.cc:482] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.269750 2835 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.275080 3123 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.277202 3123 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.266647 31120 leader_election.cc:302] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.247414 3192 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.247888 31046 raft_consensus.cc:2454] T e160777f9c924323b0223db59c1ae041 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:43.286468 31005 leader_election.cc:302] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:43.286950 1621 consensus_queue.cc:1035] T c2c9e03f3fb24c0eba0bf8f658070202 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:43.289083 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.289294 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.289673 31177 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.290031 31172 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:43.290234 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.289307 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.295773 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.295974 31120 leader_election.cc:302] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.296712 1621 raft_consensus.cc:2741] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260809 07:33:43.297091 31120 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters: 2c9de56c3fa3487c9dac990675c34be5
I20260809 07:33:43.298457 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:43.299229 30890 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 12.
I20260809 07:33:43.298584 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.302184 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.301481 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.303347 31165 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:43.303747 1621 consensus_queue.cc:1035] T 8e327743625143d2b7f941cececc7d75 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:33:43.304491 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:43.305374 31120 leader_election.cc:302] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.281571 3188 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:43.310441 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c4cb7bad240cca6d84d160e658ffe" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.305172 30890 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:43.311129 30899 raft_consensus.cc:3037] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:43.278733 2879 leader_election.cc:288] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.284199 2691 raft_consensus.cc:2726] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 07:33:43.302294 31182 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:43.295141 2904 raft_consensus.cc:2781] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 07:33:43.313982 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:43.325431 3059 raft_consensus.cc:2741] T 964c4cb7bad240cca6d84d160e658ffe P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260809 07:33:43.331377 30890 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:43.324725 31164 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
W20260809 07:33:43.338920 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 52545.3us
I20260809 07:33:43.329831 30852 leader_election.cc:302] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.300220 3193 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.341199 31006 leader_election.cc:302] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.301337 2427 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.309526 2955 raft_consensus.cc:2781] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:43.310515 3188 raft_consensus.cc:482] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:43.299113 3116 consensus_queue.cc:1046] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260809 07:33:43.344853 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:43.344792 31006 leader_election.cc:302] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.354015 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:43.364255 31057 raft_consensus.cc:2454] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:43.351907 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.278242 2835 raft_consensus.cc:504] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.352811 3104 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.350068 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcad7c2c5a034837a995628cd23c8a8b" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.365757 31058 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:43.345526 3193 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.354425 3113 raft_consensus.cc:2781] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:43.318252 3073 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.352756 2955 raft_consensus.cc:482] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:43.358587 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.378306 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:43.378645 2955 raft_consensus.cc:3037] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:43.378778 31049 raft_consensus.cc:3037] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.377907 3073 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.377830 3113 raft_consensus.cc:686] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.380823 3073 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.380971 3113 consensus_queue.cc:237] T e160777f9c924323b0223db59c1ae041 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.363752 30890 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 11.
I20260809 07:33:43.382639 3104 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.366900 30852 leader_election.cc:302] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.384577 3104 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.385067 3104 raft_consensus.cc:2781] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:43.385327 2691 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:43.385905 2691 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.387084 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:43.387770 31057 raft_consensus.cc:2379] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 7: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:43.342254 3085 raft_consensus.cc:2781] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260809 07:33:43.339910 3089 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.375850 3193 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.372287 3221 raft_consensus.cc:2781] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:43.337363 2904 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:43.342181 3112 raft_consensus.cc:2781] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:43.368966 3188 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.388947 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:43.369508 31177 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:43.388481 30911 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:43.403434 31055 raft_consensus.cc:2454] T 43743864250e4199a5862ae02b5dbed7 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:43.404565 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.404976 31005 leader_election.cc:302] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:43.390807 30899 raft_consensus.cc:2454] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:43.353049 3150 raft_consensus.cc:2781] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:43.391103 31049 raft_consensus.cc:2454] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:43.395120 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:43.345573 2879 raft_consensus.cc:482] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.409068 31051 raft_consensus.cc:2379] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 11: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:43.409116 2879 raft_consensus.cc:504] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:43.406836 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 88148.9us
I20260809 07:33:43.397523 3112 raft_consensus.cc:482] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:43.410912 3112 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.411383 2879 leader_election.cc:288] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.411684 2835 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.407281 3221 raft_consensus.cc:686] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [term 10 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:43.412798 3221 consensus_queue.cc:237] T 964c4cb7bad240cca6d84d160e658ffe P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.356916 3102 consensus_queue.cc:237] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:43.414512 2575 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:43.415375 3217 raft_consensus.cc:2781] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260809 07:33:43.415807 3217 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:43.416137 3217 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Advancing to term 13
I20260809 07:33:43.417117 2575 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:43.390991 3085 raft_consensus.cc:482] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:43.418176 2738 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.418674 2738 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.407614 3150 raft_consensus.cc:482] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:43.407589 30852 leader_election.cc:302] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:43.422729 3150 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.354378 2427 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.397923 31058 raft_consensus.cc:2454] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 10.
I20260809 07:33:43.391558 3203 raft_consensus.cc:3032] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:43.425065 3203 raft_consensus.cc:729] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.425966 3203 consensus_queue.cc:260] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.418004 3085 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Advancing to term 12
I20260809 07:33:43.427318 3203 raft_consensus.cc:3037] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.430848 31058 raft_consensus.cc:1261] T 6ba1c2ba04d7459b93ff369f6223cef6 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:43.431026 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.395273 3089 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.432824 30853 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:43.433684 30853 leader_election.cc:302] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:43.401765 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb03248d97e4a8c8ea50e8576f8cb6e" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.435060 30890 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:43.402261 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43743864250e4199a5862ae02b5dbed7" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.385411 3104 raft_consensus.cc:482] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:43.401516 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.437539 30853 leader_election.cc:302] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.436170 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.438796 31170 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:43.430748 31051 raft_consensus.cc:1261] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 07:33:43.434077 3089 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.398438 2904 raft_consensus.cc:3037] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Advancing to term 8
I20260809 07:33:43.837319 30899 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 07:33:43.844393 31175 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:43.836408 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.462824 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.849529 31175 raft_consensus.cc:595] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:43.837039 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.453505 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:43.619632 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e160777f9c924323b0223db59c1ae041" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.851320 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
W20260809 07:33:43.835109 30853 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 397625us
I20260809 07:33:43.851903 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa0c0ea2e5044208ed578990bfd5d37" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.851496 31051 raft_consensus.cc:2396] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260809 07:33:43.825847 31005 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 410095us
I20260809 07:33:43.854089 31005 leader_election.cc:302] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:43.853015 31049 raft_consensus.cc:2354] T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9.
I20260809 07:33:43.833444 30890 raft_consensus.cc:2454] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
W20260809 07:33:43.513383 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 105843us
I20260809 07:33:43.856894 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:43.854514 31181 raft_consensus.cc:2379] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:43.434573 3193 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:43.856860 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:43.859411 31170 raft_consensus.cc:1226] T 4930be41b06547ba93e04e5d1385cb3c P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:43.866171 30890 raft_consensus.cc:2454] T 9e87f17b17dd4e1fba73ed247d6b4039 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:43.858283 30911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "976fdb751bc0440a92f97b6c95314270" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.860327 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.866686 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.866293 30906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.864643 30899 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:43.424769 2427 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
W20260809 07:33:43.706614 3234 log.cc:924] Time spent T cb8db10c4bb04e86a15bcbff5ffed3d5 P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.297s user 0.003s sys 0.000s
I20260809 07:33:43.438814 3104 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.541761 3107 raft_consensus.cc:2781] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:43.555558 3073 raft_consensus.cc:3037] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:43.867478 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:43.696105 2738 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.679464 2661 leader_election.cc:288] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.876993 30906 raft_consensus.cc:2454] T b72ff16aedce4676b889395ca4d8909d P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:43.534529 3123 raft_consensus.cc:2781] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:43.884097 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbfdffc27ac46a0a18582f85cd4a00a" candidate_uuid: "28b9965e21374d26b0ac2014c9bafdff" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.513386 2691 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:43.544310 3088 raft_consensus.cc:2781] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:43.864220 3203 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:43.871767 2659 raft_consensus.cc:3037] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:43.526185 2910 raft_consensus.cc:504] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.911592 2910 leader_election.cc:288] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:43.469727 3085 raft_consensus.cc:504] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.913378 3085 leader_election.cc:288] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.650867 2955 raft_consensus.cc:504] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.877957 3136 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.922710 3136 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.924759 3136 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.896243 2738 raft_consensus.cc:2781] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:43.898099 31005 leader_election.cc:302] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:43.929615 3218 raft_consensus.cc:2781] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Leader election won for term 7
I20260809 07:33:43.930114 3218 raft_consensus.cc:686] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [term 7 LEADER]: Becoming Leader. State: Replica: a720817fc113446a8ed55e61523b280a, State: Running, Role: LEADER
I20260809 07:33:43.930876 3218 consensus_queue.cc:237] T beb03248d97e4a8c8ea50e8576f8cb6e P a720817fc113446a8ed55e61523b280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.900976 31170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:43.926262 2738 raft_consensus.cc:482] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.936285 2738 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:43.935961 3203 raft_consensus.cc:504] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.938023 3203 leader_election.cc:288] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.862990 2575 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:43.943826 2575 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.944572 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee4aa83021af4237806656803bff3e10" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:43.945713 2575 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:43.935920 3103 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.947381 3103 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.898720 3088 raft_consensus.cc:482] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 12107e38eb6d424abc4f45f16a9a1560)
I20260809 07:33:43.867739 3193 raft_consensus.cc:504] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.949317 3103 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.950723 3193 leader_election.cc:288] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.914500 31164 raft_consensus.cc:1226] T 59bd1fdaac4346aa9f2f9be0f904b408 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:43.771682 3112 raft_consensus.cc:504] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.952000 3104 raft_consensus.cc:504] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.952876 3112 leader_election.cc:288] T 9dba78a1408b4bd7a6f0c247b1176f0b P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.915580 30906 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [term 10 LEADER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:43.905349 31177 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 07:33:43.732909 3188 raft_consensus.cc:504] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.953977 3104 leader_election.cc:288] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.921898 2955 leader_election.cc:288] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:43.902667 30900 raft_consensus.cc:2454] T ee4aa83021af4237806656803bff3e10 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 8.
I20260809 07:33:43.981074 30900 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:43.942770 32197 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 10 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:43.981536 2738 raft_consensus.cc:504] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.954653 31059 raft_consensus.cc:2354] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:43.956277 3188 leader_election.cc:288] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.734683 3150 raft_consensus.cc:504] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.985281 2955 consensus_queue.cc:1046] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:43.986166 2955 raft_consensus.cc:3032] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Stepping down as leader of term 9
I20260809 07:33:43.986508 2955 raft_consensus.cc:729] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:43.987044 3150 leader_election.cc:288] T 485e0cc914a748fa957f0291e4fea329 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:43.988842 30911 raft_consensus.cc:2454] T 976fdb751bc0440a92f97b6c95314270 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:43.900552 3107 raft_consensus.cc:686] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:43.963523 30906 raft_consensus.cc:1226] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:43.963961 3073 raft_consensus.cc:2726] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 07:33:43.949370 3088 raft_consensus.cc:3037] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:44.017285 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:44.018203 31182 raft_consensus.cc:2379] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 8: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:43.677821 3217 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:43.978672 30899 raft_consensus.cc:1226] T 0fe29d891e5e411e8c46c18dcae14e66 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260809 07:33:43.979362 30896 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:44.020468 30742 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48008 (request call id 32) took 2924 ms. Trace:
I20260809 07:33:43.959493 30891 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: []
I20260809 07:33:44.021011 30742 rpcz_store.cc:276] 0809 07:33:41.125000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 07:33:41.134545 (+ 9545us) service_pool.cc:227] Handling call
0809 07:33:44.020400 (+2885855us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":113533,"reported_tablets":1}
I20260809 07:33:43.991751 31170 raft_consensus.cc:1226] T 66e26e3128404ae8824fab16033fc0ea P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Rejecting Update request from peer 2c9de56c3fa3487c9dac990675c34be5 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:43.994552 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:44.022655 31181 raft_consensus.cc:2379] T 7635c2a6de6446b294b707dbb8bf244a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 in current term 9: Already voted for candidate 12107e38eb6d424abc4f45f16a9a1560 in this term.
I20260809 07:33:43.995215 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:44.023523 31175 raft_consensus.cc:2454] T b2be7407d0cf4ac2a1a0a1e6525c9bac P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 8.
I20260809 07:33:43.996183 31177 raft_consensus.cc:1261] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260809 07:33:43.973261 2904 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.028338 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:44.033308 3120 raft_consensus.cc:482] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.033887 3103 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 07:33:44.034642 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 555779us
I20260809 07:33:44.034411 3103 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.035290 31006 leader_election.cc:302] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:44.035989 3248 raft_consensus.cc:2781] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 07:33:44.036391 3248 raft_consensus.cc:482] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:44.036558 3103 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.036723 3248 raft_consensus.cc:3037] T 7635c2a6de6446b294b707dbb8bf244a P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:44.042414 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:44.043067 31182 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:44.044454 31182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:44.045012 31182 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 10 FOLLOWER]: Advancing to term 12
I20260809 07:33:44.046645 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:44.047302 31177 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:44.048147 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae547589201f452eb2ae83ea11689f7e" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:44.051265 31055 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.052464 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:44.016780 3123 raft_consensus.cc:686] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:44.056824 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:44.063820 31177 raft_consensus.cc:3037] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.057791 31175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:44.018710 3221 raft_consensus.cc:482] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:44.149969 31175 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.150519 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e327743625143d2b7f941cececc7d75" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:44.152452 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b72ff16aedce4676b889395ca4d8909d" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:44.153137 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ccd6862a094ecba07b495d4c772721" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:44.055084 3089 raft_consensus.cc:482] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:44.291051 31049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e580429091da41448485d00896fcf32e" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:44.041703 3240 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:43.987264 2955 consensus_queue.cc:260] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } attrs { replace: true } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } peers { permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false } }
I20260809 07:33:44.322230 1621 consensus_queue.cc:1035] T 0e6264779cf74c14bc8c18901dfe6a09 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260809 07:33:44.322690 31120 leader_election.cc:302] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:44.063287 2879 raft_consensus.cc:482] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:44.330437 31120 leader_election.cc:302] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:44.070945 3104 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.321658 30745 catalog_manager.cc:5495] T aeccd39240c34b5fabec551b21206a7f P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:44.335776 31177 raft_consensus.cc:2454] T 0a72c5de04d44d51b1e44b8be3e471b2 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:44.336459 31182 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 12107e38eb6d424abc4f45f16a9a1560 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 12.
I20260809 07:33:44.327139 31057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4930be41b06547ba93e04e5d1385cb3c" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:44.296790 31046 raft_consensus.cc:1226] T fcad7c2c5a034837a995628cd23c8a8b P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260809 07:33:44.338176 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e87f17b17dd4e1fba73ed247d6b4039" candidate_uuid: "12107e38eb6d424abc4f45f16a9a1560" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:44.338956 31046 raft_consensus.cc:2379] T 9e87f17b17dd4e1fba73ed247d6b4039 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 in current term 9: Already voted for candidate a720817fc113446a8ed55e61523b280a in this term.
I20260809 07:33:44.342713 31177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
I20260809 07:33:44.033885 3120 raft_consensus.cc:504] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.296422 31055 raft_consensus.cc:2454] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:44.349532 31048 raft_consensus.cc:1226] T ae547589201f452eb2ae83ea11689f7e P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9. Ops: []
I20260809 07:33:44.351379 31057 raft_consensus.cc:1226] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260809 07:33:44.289166 2738 leader_election.cc:288] T 5f1326c9441f41f4ad3abaac9ca2b2ec P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.292033 2427 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.356894 31054 raft_consensus.cc:1226] T 3bbfdffc27ac46a0a18582f85cd4a00a P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:44.323226 3189 raft_consensus.cc:482] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.295738 3252 raft_consensus.cc:482] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:44.370609 3252 raft_consensus.cc:504] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.372601 3252 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:44.296051 2669 raft_consensus.cc:482] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.328841 2575 raft_consensus.cc:482] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.373255 2669 raft_consensus.cc:504] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.373395 2575 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.375236 2669 leader_election.cc:288] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.362040 31054 raft_consensus.cc:1261] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Refusing update from remote peer 12107e38eb6d424abc4f45f16a9a1560: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260809 07:33:44.363109 32141 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.28.87.67:44107 timed out after 2.880s (SENT) [suppressed 3 similar messages]
I20260809 07:33:44.354674 2879 raft_consensus.cc:504] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.363430 2955 raft_consensus.cc:3037] T 3bbfdffc27ac46a0a18582f85cd4a00a P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:44.363626 3202 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.340921 3221 raft_consensus.cc:504] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.419476 3202 raft_consensus.cc:504] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.364661 3188 raft_consensus.cc:482] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.365898 3189 raft_consensus.cc:504] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.337543 3240 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.336748 3089 raft_consensus.cc:504] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.427290 3240 leader_election.cc:288] T b4960950e96549f499238d58397046ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.368788 3218 raft_consensus.cc:482] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:44.427920 3218 raft_consensus.cc:504] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.380028 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba1c2ba04d7459b93ff369f6223cef6" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.429884 3218 leader_election.cc:288] T 401ec3203e7b44faae0c513339cc7317 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.379474 3242 raft_consensus.cc:482] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:44.379938 31046 raft_consensus.cc:1226] T e8bb7f31be3d4e2faf59b0f97f6a803d P a720817fc113446a8ed55e61523b280a [term 7 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 6. Current term is 7. Ops: []
I20260809 07:33:44.378464 3166 raft_consensus.cc:482] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.338714 2904 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:44.357636 2891 raft_consensus.cc:482] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:44.352285 3251 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.379068 2548 raft_consensus.cc:482] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.379608 30852 leader_election.cc:302] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5; no voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a
I20260809 07:33:44.442377 31045 raft_consensus.cc:2454] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c9de56c3fa3487c9dac990675c34be5 in term 9.
I20260809 07:33:44.377120 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.446907 30891 raft_consensus.cc:2454] T 5df12775e0504b2da44e2fa37c5408ce P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:44.447943 30891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a72c5de04d44d51b1e44b8be3e471b2" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:44.355800 1621 raft_consensus.cc:2781] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:44.448760 1621 raft_consensus.cc:686] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.449594 1621 consensus_queue.cc:237] T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.355258 30911 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e327743625143d2b7f941cececc7d75"
dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:54230
I20260809 07:33:44.450730 30911 raft_consensus.cc:595] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:44.451068 30911 raft_consensus.cc:603] T 8e327743625143d2b7f941cececc7d75 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:44.355640 31120 leader_election.cc:302] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
W20260809 07:33:44.453081 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 97519.5us
I20260809 07:33:44.352509 3235 consensus_queue.cc:1046] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260809 07:33:44.454377 31177 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e160777f9c924323b0223db59c1ae041"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:44.455042 31177 raft_consensus.cc:595] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:44.455411 31177 raft_consensus.cc:603] T e160777f9c924323b0223db59c1ae041 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:44.455749 3252 consensus_queue.cc:1046] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:33:44.456647 3252 raft_consensus.cc:3032] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Stepping down as leader of term 9
I20260809 07:33:44.456956 3252 raft_consensus.cc:729] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.458228 31177 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e580429091da41448485d00896fcf32e"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:44.458837 31177 raft_consensus.cc:595] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Received request to transfer leadership to TS a720817fc113446a8ed55e61523b280a
I20260809 07:33:44.458499 3059 raft_consensus.cc:3032] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:44.459514 3059 raft_consensus.cc:729] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.460397 3059 consensus_queue.cc:260] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.461633 3059 raft_consensus.cc:3037] T beb03248d97e4a8c8ea50e8576f8cb6e P 12107e38eb6d424abc4f45f16a9a1560 [term 6 FOLLOWER]: Advancing to term 7
I20260809 07:33:44.424203 3221 leader_election.cc:288] T 5e48f193f20844a4ac619300cfaac1bd P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.383567 2427 raft_consensus.cc:504] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.464155 2427 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.424366 3188 raft_consensus.cc:504] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.466140 3188 leader_election.cc:288] T 4d8f75ef38314f3185fbcd1e0b1df988 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.427340 3089 leader_election.cc:288] T 87ac258020ac4e29b08523647d3ab361 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.400247 2948 raft_consensus.cc:482] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a720817fc113446a8ed55e61523b280a)
I20260809 07:33:44.382086 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "723b3b7ed7b749d3b4b448d22577a8e7" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:44.467106 30900 raft_consensus.cc:3037] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Advancing to term 12
I20260809 07:33:44.467021 2948 raft_consensus.cc:504] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.469007 2948 leader_election.cc:288] T 08675f2b92e541ecba618784caa19bb3 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.455260 3235 raft_consensus.cc:2781] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 07:33:44.433393 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11936edbc86a4c4fbf8e08748a48d8cb" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.470516 3235 raft_consensus.cc:686] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.470602 30899 raft_consensus.cc:2454] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:44.471467 3235 consensus_queue.cc:237] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.346158 3104 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.457759 3252 consensus_queue.cc:260] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.475318 3104 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.375308 2575 leader_election.cc:288] T b4960950e96549f499238d58397046ce P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:44.413879 2738 raft_consensus.cc:482] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.435145 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dba78a1408b4bd7a6f0c247b1176f0b" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:44.435812 31049 raft_consensus.cc:3037] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.454372 31120 leader_election.cc:302] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:44.455538 3223 consensus_queue.cc:1046] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260809 07:33:44.477662 3223 raft_consensus.cc:3032] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:44.478008 3223 raft_consensus.cc:729] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.434087 30745 catalog_manager.cc:5495] T ee4aa83021af4237806656803bff3e10 P 12107e38eb6d424abc4f45f16a9a1560 reported cstate change: term changed from 6 to 7, leader changed from 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67) to 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65). New cstate: current_term: 7 leader_uuid: "2c9de56c3fa3487c9dac990675c34be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } } }
I20260809 07:33:44.345224 3205 raft_consensus.cc:2781] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:44.478853 3205 raft_consensus.cc:482] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
I20260809 07:33:44.479272 3205 raft_consensus.cc:3037] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.478768 3223 consensus_queue.cc:260] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:44.443418 30852 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.66:36075, user_credentials={real_user=slave}} blocked reactor thread for 63865.7us
I20260809 07:33:44.480022 3223 raft_consensus.cc:3037] T ae547589201f452eb2ae83ea11689f7e P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.480316 30852 leader_election.cc:302] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters: 12107e38eb6d424abc4f45f16a9a1560
I20260809 07:33:44.481734 30852 leader_election.cc:398] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 8 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c9de56c3fa3487c9dac990675c34be5 for earlier term 8. Current term is 9.
I20260809 07:33:44.443982 2835 raft_consensus.cc:3037] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Advancing to term 11
I20260809 07:33:44.391153 31055 raft_consensus.cc:1226] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [term 11 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260809 07:33:44.378039 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.459201 31177 raft_consensus.cc:603] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 07:33:44.484293 30896 raft_consensus.cc:2454] T 7635c2a6de6446b294b707dbb8bf244a P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:44.485069 2575 raft_consensus.cc:2781] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:44.485572 2575 raft_consensus.cc:686] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.486433 2575 consensus_queue.cc:237] T e580429091da41448485d00896fcf32e P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.487396 3104 raft_consensus.cc:2781] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 07:33:44.487911 3104 raft_consensus.cc:686] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [term 10 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:44.489452 31177 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ee4aa83021af4237806656803bff3e10"
dest_uuid: "12107e38eb6d424abc4f45f16a9a1560"
mode: GRACEFUL
new_leader_uuid: "a720817fc113446a8ed55e61523b280a"
from {username='slave'} at 127.0.0.1:44244
I20260809 07:33:44.486151 2738 raft_consensus.cc:504] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:44.497002 30724 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 3bbfdffc27ac46a0a18582f85cd4a00a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260809 07:33:44.475495 3252 raft_consensus.cc:3037] T fcad7c2c5a034837a995628cd23c8a8b P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:44.498559 30904 raft_consensus.cc:3037] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.488715 3104 consensus_queue.cc:237] T fcad7c2c5a034837a995628cd23c8a8b P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:44.503167 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 48920us
I20260809 07:33:44.504477 3247 consensus_queue.cc:1046] T 4d8f75ef38314f3185fbcd1e0b1df988 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 07:33:44.505695 31120 leader_election.cc:302] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, 2c9de56c3fa3487c9dac990675c34be5; no voters:
I20260809 07:33:44.506642 2904 consensus_queue.cc:1046] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 07:33:44.509513 30900 raft_consensus.cc:2454] T 723b3b7ed7b749d3b4b448d22577a8e7 P 2c9de56c3fa3487c9dac990675c34be5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 12.
I20260809 07:33:44.513593 2835 raft_consensus.cc:2726] T c2c9e03f3fb24c0eba0bf8f658070202 P 2c9de56c3fa3487c9dac990675c34be5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 07:33:44.517005 30904 raft_consensus.cc:2454] T 9dba78a1408b4bd7a6f0c247b1176f0b P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:44.494031 31049 raft_consensus.cc:2454] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12107e38eb6d424abc4f45f16a9a1560 in term 9.
I20260809 07:33:44.352838 31175 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:44.352061 3107 consensus_queue.cc:237] T afa0c0ea2e5044208ed578990bfd5d37 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.529680 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485e0cc914a748fa957f0291e4fea329" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:44.530676 30904 raft_consensus.cc:3037] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.439103 3217 leader_election.cc:288] T 202d19fadaa547209afb8e0109ff54e8 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 13 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.430306 3088 raft_consensus.cc:504] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.540936 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1326c9441f41f4ad3abaac9ca2b2ec" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:44.541610 30896 raft_consensus.cc:3037] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:44.414198 3123 consensus_queue.cc:237] T 0a72c5de04d44d51b1e44b8be3e471b2 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.545691 30852 leader_election.cc:302] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c9de56c3fa3487c9dac990675c34be5, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:44.440955 2891 raft_consensus.cc:504] T dd3163efce7d454fb409a007f6ee93ef P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.439769 3166 raft_consensus.cc:504] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
W20260809 07:33:44.532348 32141 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.67:44107, user_credentials={real_user=slave}} blocked reactor thread for 192251us
I20260809 07:33:44.558588 30904 raft_consensus.cc:2454] T 485e0cc914a748fa957f0291e4fea329 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 9.
I20260809 07:33:44.557384 30909 raft_consensus.cc:1226] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Rejecting Update request from peer 12107e38eb6d424abc4f45f16a9a1560 for earlier term 9. Current term is 10. Ops: []
I20260809 07:33:44.559039 30900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.562157 31054 raft_consensus.cc:1261] T 31954ed4d71d4a6796aa3c4bb2ded761 P a720817fc113446a8ed55e61523b280a [term 10 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 07:33:44.521814 2548 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.565490 1621 raft_consensus.cc:2781] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:44.557864 30896 raft_consensus.cc:2454] T 5f1326c9441f41f4ad3abaac9ca2b2ec P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 10.
I20260809 07:33:44.570916 1621 raft_consensus.cc:686] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 LEADER]: Becoming Leader. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.569378 30900 raft_consensus.cc:2396] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260809 07:33:44.569272 31006 leader_election.cc:302] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:44.440568 3242 raft_consensus.cc:504] T 43645669985b45a0b7e6d18147b28424 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.571399 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7635c2a6de6446b294b707dbb8bf244a" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:44.555965 2835 raft_consensus.cc:2781] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 07:33:44.519704 3251 raft_consensus.cc:504] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.580230 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8bb7f31be3d4e2faf59b0f97f6a803d" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.584513 30899 raft_consensus.cc:2454] T e8bb7f31be3d4e2faf59b0f97f6a803d P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 7.
I20260809 07:33:44.585255 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401ec3203e7b44faae0c513339cc7317" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.588289 31006 leader_election.cc:302] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:44.597071 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ac258020ac4e29b08523647d3ab361" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.597357 30904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "202d19fadaa547209afb8e0109ff54e8" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5"
I20260809 07:33:44.597893 30904 raft_consensus.cc:3037] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 12 FOLLOWER]: Advancing to term 13
W20260809 07:33:44.598009 31120 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.28.87.65:33749, user_credentials={real_user=slave}} blocked reactor thread for 92375.3us
I20260809 07:33:44.599891 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4960950e96549f499238d58397046ce" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.600471 30899 raft_consensus.cc:2454] T b4960950e96549f499238d58397046ce P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 8.
I20260809 07:33:44.600625 30891 raft_consensus.cc:1261] T 964c4cb7bad240cca6d84d160e658ffe P 2c9de56c3fa3487c9dac990675c34be5 [term 10 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 07:33:44.601006 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e48f193f20844a4ac619300cfaac1bd" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.601282 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c9e03f3fb24c0eba0bf8f658070202" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.602005 30896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8f75ef38314f3185fbcd1e0b1df988" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c9de56c3fa3487c9dac990675c34be5" is_pre_election: true
I20260809 07:33:44.602475 30899 raft_consensus.cc:1261] T beb03248d97e4a8c8ea50e8576f8cb6e P 2c9de56c3fa3487c9dac990675c34be5 [term 7 FOLLOWER]: Refusing update from remote peer a720817fc113446a8ed55e61523b280a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 07:33:44.602576 30896 raft_consensus.cc:2396] T 4d8f75ef38314f3185fbcd1e0b1df988 P 2c9de56c3fa3487c9dac990675c34be5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260809 07:33:44.569574 3188 raft_consensus.cc:482] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:33:44.605295 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df12775e0504b2da44e2fa37c5408ce" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:44.569732 3120 leader_election.cc:288] T c2c9e03f3fb24c0eba0bf8f658070202 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.596287 3258 raft_consensus.cc:2781] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:44.569446 3202 leader_election.cc:288] T 11936edbc86a4c4fbf8e08748a48d8cb P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.607625 1621 consensus_queue.cc:237] T 976fdb751bc0440a92f97b6c95314270 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.587663 3247 raft_consensus.cc:3032] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 07:33:44.613693 30904 raft_consensus.cc:2454] T 202d19fadaa547209afb8e0109ff54e8 P 2c9de56c3fa3487c9dac990675c34be5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720817fc113446a8ed55e61523b280a in term 13.
I20260809 07:33:44.614163 3266 consensus_queue.cc:1035] T 6ba1c2ba04d7459b93ff369f6223cef6 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:44.616518 32197 raft_consensus.cc:1261] T 3bbfdffc27ac46a0a18582f85cd4a00a P 28b9965e21374d26b0ac2014c9bafdff [term 10 FOLLOWER]: Refusing update from remote peer 2c9de56c3fa3487c9dac990675c34be5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260809 07:33:44.604262 3240 raft_consensus.cc:2781] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 07:33:44.619321 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17576a737982459aa79301ce4f7eb096" candidate_uuid: "a720817fc113446a8ed55e61523b280a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "12107e38eb6d424abc4f45f16a9a1560" is_pre_election: true
I20260809 07:33:44.600389 2835 raft_consensus.cc:686] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Leader. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:44.622269 31006 leader_election.cc:302] T 0a72c5de04d44d51b1e44b8be3e471b2 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:44.563357 3205 raft_consensus.cc:504] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.623914 31181 raft_consensus.cc:2396] T 17576a737982459aa79301ce4f7eb096 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a720817fc113446a8ed55e61523b280a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260809 07:33:44.625715 31006 leader_election.cc:302] T 723b3b7ed7b749d3b4b448d22577a8e7 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12107e38eb6d424abc4f45f16a9a1560, a720817fc113446a8ed55e61523b280a; no voters:
I20260809 07:33:44.626509 31120 leader_election.cc:398] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Vote denied by peer a720817fc113446a8ed55e61523b280a (127.28.87.66:36075) with higher term. Message: Invalid argument: T afa0c0ea2e5044208ed578990bfd5d37 P a720817fc113446a8ed55e61523b280a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 12107e38eb6d424abc4f45f16a9a1560 for earlier term 8. Current term is 9.
I20260809 07:33:44.626816 31120 leader_election.cc:401] T afa0c0ea2e5044208ed578990bfd5d37 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260809 07:33:44.621469 2879 leader_election.cc:288] T e580429091da41448485d00896fcf32e P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.629909 3189 leader_election.cc:288] T 8e327743625143d2b7f941cececc7d75 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.628682 3069 consensus_queue.cc:1046] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260809 07:33:44.631881 31048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2be7407d0cf4ac2a1a0a1e6525c9bac" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a" is_pre_election: true
I20260809 07:33:44.632107 3069 raft_consensus.cc:3032] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Stepping down as leader of term 9
I20260809 07:33:44.632671 3069 raft_consensus.cc:729] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:44.626173 3102 consensus_queue.cc:1035] T 31954ed4d71d4a6796aa3c4bb2ded761 P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260809 07:33:44.638306 3266 consensus_queue.cc:1035] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
I20260809 07:33:44.639127 2835 consensus_queue.cc:237] T ae547589201f452eb2ae83ea11689f7e P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.641026 2738 leader_election.cc:288] T 0e6264779cf74c14bc8c18901dfe6a09 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.642354 3192 consensus_queue.cc:1035] T 3bbfdffc27ac46a0a18582f85cd4a00a P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9965e21374d26b0ac2014c9bafdff" member_type: VOTER last_known_addr { host: "127.28.87.68" port: 39299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 07:33:44.633998 3069 consensus_queue.cc:260] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.644440 3069 raft_consensus.cc:3037] T 59bd1fdaac4346aa9f2f9be0f904b408 P 2c9de56c3fa3487c9dac990675c34be5 [term 9 FOLLOWER]: Advancing to term 10
I20260809 07:33:44.644961 31058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fe29d891e5e411e8c46c18dcae14e66" candidate_uuid: "2c9de56c3fa3487c9dac990675c34be5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a720817fc113446a8ed55e61523b280a"
I20260809 07:33:44.645586 31058 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.646116 3088 leader_election.cc:288] T 43743864250e4199a5862ae02b5dbed7 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 10 election: Requested vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.648792 3166 leader_election.cc:288] T 17576a737982459aa79301ce4f7eb096 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.650076 3205 leader_election.cc:288] T 9e87f17b17dd4e1fba73ed247d6b4039 P 12107e38eb6d424abc4f45f16a9a1560 [CANDIDATE]: Term 9 election: Requested vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), a720817fc113446a8ed55e61523b280a (127.28.87.66:36075)
I20260809 07:33:44.599380 2948 consensus_queue.cc:1046] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260809 07:33:44.621860 3240 raft_consensus.cc:482] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:44.656015 2948 raft_consensus.cc:3032] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 07:33:44.656543 3240 raft_consensus.cc:3037] T 11936edbc86a4c4fbf8e08748a48d8cb P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.656680 2948 raft_consensus.cc:729] T 4930be41b06547ba93e04e5d1385cb3c P 2c9de56c3fa3487c9dac990675c34be5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c9de56c3fa3487c9dac990675c34be5, State: Running, Role: LEADER
I20260809 07:33:44.611232 3258 raft_consensus.cc:482] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:33:44.658567 3258 raft_consensus.cc:3037] T 5df12775e0504b2da44e2fa37c5408ce P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.609076 3188 raft_consensus.cc:504] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.660246 2548 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P 2c9de56c3fa3487c9dac990675c34be5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a720817fc113446a8ed55e61523b280a (127.28.87.66:36075), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.661171 3188 leader_election.cc:288] T b2be7407d0cf4ac2a1a0a1e6525c9bac P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.617612 3247 raft_consensus.cc:729] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 12107e38eb6d424abc4f45f16a9a1560, State: Running, Role: LEADER
I20260809 07:33:44.674940 3251 leader_election.cc:288] T cfb2fb324ee14e2084a95783b59cab00 P a720817fc113446a8ed55e61523b280a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c9de56c3fa3487c9dac990675c34be5 (127.28.87.65:33749), 12107e38eb6d424abc4f45f16a9a1560 (127.28.87.67:44107)
I20260809 07:33:44.675715 3247 consensus_queue.cc:260] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c9de56c3fa3487c9dac990675c34be5" member_type: VOTER last_known_addr { host: "127.28.87.65" port: 33749 } } peers { permanent_uuid: "a720817fc113446a8ed55e61523b280a" member_type: VOTER last_known_addr { host: "127.28.87.66" port: 36075 } } peers { permanent_uuid: "12107e38eb6d424abc4f45f16a9a1560" member_type: VOTER last_known_addr { host: "127.28.87.67" port: 44107 } }
I20260809 07:33:44.677402 3247 raft_consensus.cc:3037] T 0fe29d891e5e411e8c46c18dcae14e66 P 12107e38eb6d424abc4f45f16a9a1560 [term 8 FOLLOWER]: Advancing to term 9
I20260809 07:33:44.667133 3189 raft_consensus.cc:482] T 3c486ed5f0d9423992400dfce71ce70f P a720817fc113446a8ed55e61523b280a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c9de56c3fa3487c9dac990675c34be5)
W20260809 07:33:44.671449 3265 log.cc:924] Time spent T b72ff16aedce4676b889395ca4d8909d P 12107e38eb6d424abc4f45f16a9a1560: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
W20260809 07:33:44.672008 31006 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote ->